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ABSTRACT 



This final report/doctoral dissertation describes activities 
and accomplishments of Project LIFT (Looking at Intervention Factors with 
Teachers) , which assessed the relationships among classroom intervention 
acceptability, integrity, and effectiveness. Two studies were conducted. The 
first involved observing interventions implemented in 10 Iowa elementary 
classroom settings. In the second study, 350 elementary teachers in 11 states 
were surveyed with a project-developed questionnaire concerning their 
experiences and perceptions of interventions they have implemented with 
students. Findings from both studies indicated that teachers tended to 
receive assistance when developing an intervention, but then implemented the 
intervention alone. Individualized intervention plans rarely described 
specific steps to be completed. Most teachers rated the interventions from 
moderate to very acceptable in both studies and implementation integrity was 
found to be high in the observation study. Teachers responding to the survey 
indicated more use of formal efforts to maintain intervention integrity than 
teachers in the observation study actually used. Appended are interview forms 
used in the observational study, the questionnaire used in the national 
survey, and factor analysis results. (DB) 
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Increasing emphasis is being placed on providing educational interventions for children 
with learning and behavioral problems in the least restrictive environment (LRE). Several of the 
factors suggested as important in intervention implementation include intervention acceptability, 
integrity, and effectiveness. Most school-based intervention plans do not include assessment of 
these factors. Little research has been completed to examine interventions in applied settings to 
determine if the hypothesized relationship between the components exists. 

The overall purpose of this project was to assess the relationships between intervention 
acceptability, intervention integrity, and the effectiveness of classroom interventions using two 
studies. The first study involved observing interventions as they were implemented in elementary 
classroom settings. The second study used a survey to ask elementary teachers in 1 1 states about 
their experiences and perceptions of interventions they have implemented for students. 

Findings from both studies indicated that teachers tended to receive assistance when 
developing the interventions, but then implemented the interventions alone. In general, 
individualized interventions plans did not describe specific steps to be completed as part of the 
intervention. Most teachers rated the interventions from moderate to very acceptable in both 
studies. Implementation integrity was found to be high in the observation study. Teachers in the 
survey indicated more use of formal efforts to maintain intervention integrity than did teachers in 
the observation study. Further findings and implications are also discussed in the attached 



dissertation. 



11 



Iowa State University 



This is to certify that the Doctoral dissertation of 
Gayle Joanne Luze 

has met the dissertation requirements of Iowa State University 



Major Professor 



For the Major Program 



For the Graduate College 




r— 

v> 



Ill 



TABLE OF CONTENTS 

LIST OF FIGURES v 

LIST OF TABLES vi 

ABSTRACT vii 

GENERAL INTRODUCTION 1 

LITERATURE REVIEW 5 

PART I. LOCAL OBSERVATION STUDY 3 1 

INTRODUCTION 32 

METHODOLOGY 34 

RESULTS 44 

DISCUSSION 59 

PART II. NATIONAL SURVEY STUDY 67 

INTRODUCTION 68 

METHODOLOGY 70 

RESULTS 76 

DISCUSSION 82 

GENERAL CONCLUSION 87 

APPENDIX A: INITIAL INTER VIEWAND CLOSING TEACHER INTERVIEW 94 

APPENDIX B : INTIIAL INTERVENTION COMPARISON SCALE (IICS) AND 

CLOSING INTERVENTION COMPARISON SCALE (CICS) 1 02 

APPENDIX C: INTERVENTION OBSERVATION FORM (IOF) 1 1 0 




6 



APPENDIX D: NATIONAL QUESTIONNAIRE 
APPENDIX E: FACTOR ANALYSIS 
REFERENCES 




( 



V 



LIST OF FIGURES 



Figure 1 . Graphed means and standard deviations of IICS items. 5 1 

Figure 2. Graphed means and standard deviations of CICS items. 52 

Figure 3. Examples of Intervention Checklists (IC’s). 54 

Figure 4. Factor analysis scree plot. 126 




8 



VI 



LIST OF TABLES 



Table 1. Means and standard deviations for IICS and CICS items. 50 

Table 2. t test of IICS and CICS items. 53 

Table 3 . Respondent background information. 7 1 

Table 4. Descriptions of classrooms. 72 

Table 5. Comparisons of teachers in Iowa and national samples on intervention guidelines, 

use, and documentation. 79 

Table 6. Summary of regression analyses for related to intervention acceptability and 

integrity. 81 

Table 7. Survey Intervention Rating scale item correlations. 123 

Table 8. Varimax factor matrix. 127 




9 



Vll 



ABSTRACT 



Increasing emphasis is being placed on providing educational interventions for 
children with learning and behavioral problems in the least restrictive environment (LRE). 
This results in greater pressure on teachers, who often have had little or no specialized 
training in classroom management and individualized educational needs (Peterson & Casey, 
1991). Efforts to assist teachers working with students having difficulty usually involve 
teachers using a problem-solving process to develop individualized interventions. However, 
little applied research has been conducted to confirm the critical factors important in 
developing and implementing interventions to achieve positive student outcomes in school 
settings. 

Several of the factors suggested to be critical in intervention implementation include 
intervention acceptability, integrity, and effectiveness. Most school-based intervention plans 
do not include assessment of these factors. Both intervention acceptability and integrity are 
hypothesized to influence intervention implementation and effectiveness. Little research has 
been completed to examine interventions in applied settings to determine if the hypothesized 
relationships between these intervention factors exists. 

The overall purpose of this project was to assess the relationships between 
intervention acceptability, intervention integrity, and the effectiveness of classroom 
interventions using two studies. The first study involved observing interventions as they were 
implemented in elementary classroom settings. The second study used a survey to ask 
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elementary teachers in 1 1 states about their experiences with and perceptions about 
interventions they have implemented for students. 

Findings from both studies indicated that teachers tended to receive assistance in 
developing the interventions, but then implemented the interventions alone. In general, plans 
developed for the interventions did not describe the specific steps to be completed as part of 
the interventions. Most teachers in both studies rated the interventions they implemented 
from moderate to very acceptable. Implementation integrity was also found to be high in the 
observation study. Teachers responding to the survey indicated more use of formal efforts to 
maintain intervention integrity than teachers in the observation study actually used. Further 
findings and implications are also discussed. 
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GENERAL INTRODUCTION 



Introduction 

Increasing emphasis is being placed on providing educational interventions for 
children with learning and behavioral problems in the least restrictive environment (LRE). 
This results in greater pressure on teachers, who have often had little or no specialized 
training in classroom management and individualized educational needs (Peterson & Casey, 
1991). Efforts to assist teachers working with students having difficulty in school usually 
involve teachers working within a problem-solving process to develop individualized 
interventions. However, little applied research has been conducted to confirm the critical 
factors important in developing and implementing interventions to achieve positive student 
outcomes in school settings. 

Several factors have been identified as critical in intervention implementation. These 
include intervention acceptability and implementation integrity (Witt & Elliott, 1985; 
Shapiro, 1987). Intervention acceptability refers to judgments made by potential consumers 
about the fairness, appropriateness, and intrusiveness of the intervention (Kazdin, 1980a; 

Witt & Elliott, 1985). No matter how appropriate or effective an intervention may be, if the 
people expected to implement an intervention do not view it as acceptable, the likelihood 
they will do so is drastically reduced. 

Intervention integrity is defined as the degree to which an intervention is 
implemented as designed (Gresham, 1 989). When intervention integrity is not assessed, no 
reliable determination can be made as to whether the intervention was responsible for student 
change (Tilly & Flugum, 1995). However, most actual intervention plans do not include 
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assessment of intervention integrity during or after implementation. In fact, most intervention 
research includes no assessment of implementation integrity (Gresham, Gansle, & Noell, 
1993; Gresham, Gansle, Noell, Cohen, & Rosenblum, 1993; Peterson, Homer & Wonderlich, 
1982). 

Both intervention acceptability and integrity are hypothesized to influence 
intervention implementation and effectiveness. Effective interventions are more likely to lead 
to better student outcomes. Resulting student outcomes are compared to goals or expectations 
set for them by school personnel and parents. While this link has been discussed in the 
literature, little research has been done to examine interventions in applied settings in an 
effort to determine if the hypothesized relationships between the components exists (Watson, 
Sterling, & McDade, 1997). 

The overall purpose of this project was to assess the relationships between 
intervention acceptability, intervention integrity, and the effectiveness of classroom 
interventions. To accomplish this goal, interventions developed for young elementary 
students experiencing learning and behavioral concerns implemented in general classroom 
settings were examined. 



Research Questions 

This project was designed to examine factors that influence acceptability of 
interventions, integrity of intervention implementation, and effectiveness of interventions in 
school settings. The project included two studies, one a local study using interviews and 
observations of interventions, the other study was a questionnaire sent to teachers in a 
number of states. Specifically, the following questions were addressed by this project: 
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(1) What is the relationship between teacher ratings of intervention acceptability and 
intervention integrity? 

(2) How well can teachers accurately measure intervention integrity as part of the 
intervention implementation process? 

(3) What is the relationship between intervention integrity and student outcome? 

(4) What factors influence intervention implementation in general education 
classrooms? 

(5) How are intervention acceptability and intervention integrity measured and 
evaluated in actual classroom practice? 

To answer question one, data were collected using interviews and scale 
questionnaires to complete correlational analyses from both studies. Correlational data were 
gathered to answer question two using intervention checklists developed as part of the 
observation study. Question three involved gathering descriptive and correlational data 
through interviews and observations as part of the observation study. Descriptive analyses of 
interview and questionnaire data from both studies were utilized to answer questions four and 
five. 

It was hypothesized that higher levels of acceptability will be related to higher levels 
of intervention integrity (question 1). It was further hypothesized that higher levels of 
intervention integrity will be related to more positive student outcomes (question 3). No 
hypotheses were developed for questions 2, 4, and 5, as these are more exploratory questions. 




14 



4 



Dissertation Organization 

In an effort to address the prepared research questions, two related projects were undertaken. 
The first project was a local observation study designed to examine actual interventions as 
they were implemented in school settings. The second project was a national survey study 
that examined issues identified in the local study to deter min e if they inf luenced 
interventions in similar ways across different geographic areas. As a result, the dissertation 
will be divided into several sections. The following subsection will contain a literature 
review applicable to both projects, as both projects are examining the same issues. Each 
project will be discussed in separate sections, followed by a general conclusion section 
discussing the relationships between the information gained from each project. 
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LITERATURE REVIEW 

/ Introduction 

Reauthorization of the Individuals with Disabilities Education Act (IDEA) and other 
policy initiatives are part of the impetus for increasing attention being focused on the needs 
of children with special needs in public schools. IDEA requires children receive a free and 
appropriate education (FAPE) in the least restrictive environment (LRE). The least restrictive 
environment is one in which services are provided in a setting as similar as possible to the 
educational environment of children without disabilities. Services in the least restrictive 
environment are more than just good policy; children have been found to learn both academic 
and social skills better in typical classroom environments (Graden, Zins, Curtis, & Cobb, 
1988; Peterson & Casey, 1991). In addition to more students receiving special education in 
general education settings, more students who struggle with the general curriculum and who 
are considered at-risk for school failure are receiving individualized assistance in their 
general classrooms (Graden, Casey, & Christenson, 1985; Graden, et al., 1988). 

Teachers in general classrooms often have had little or no training in classroom 
management or in dealing with children who experience problems in school (Bear, 1990; 
Berliner, 1988). As a result, teachers are often less willing to engage in interventions to help 
the student remain in the general class setting (Peterson & Casey, 1991). However, teachers 
are frequently not given an option to refuse to modify curricula and implement specialized 
interventions. Legal requirements, as well as increasing parent insistence that their children 
be served in general classrooms, have placed greater emphasis on the need to assist teachers 
develop additional strategies to meet these students' needs in general class settings (Myles & 
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Simpson, 1992). In addition, pressure is being placed on schools to use resources as wisely as 
possible to achieve the greatest gain. As a result, interest in determining the most effective 
interventions to address student need is growing. 

One method used to develop interventions designed to address students’ individual 
needs is a problem-solving approach. Problem-solving is a structured method used to develop 
interventions designed to meet individual student needs. While a number of approaches can 
be used, those usually used in education include behavioral and collaborative problem- 
solving (Telzrow, 1995; Tilly & Flugum, 1995). In general, problem-solving approaches 
include describing the identified problem, generating and selecting possible solutions, testing 
the chosen alternative, and evaluating effectiveness of the intervention once it has been 
implemented (Deno, 1995). 

Problem solving can occur with one person, but in educational settings, it more often 
includes two or more people. Those involved in the problem-solving process can be support 
personnel (e.g., school psychologists, special education teachers/consultants) or building- 
based teacher assistance teams. The resulting interventions can focus on students in need of 
additional compensatory or remedial assistance, as well as those requiring special education 
programming, and can occur in a general classroom or alternative setting. 

In order to develop effective interventions using a problem solving approach, the 
student’s skills must be assessed. Information gathered during this assessment will be used to 
determine the area(s) of need. This information can be used to generate and select possible 
interventions, as well as develop an appropriate method to measure progress. It is also 
important that the problem-solving process itself be evaluated to ensure reliable and valid 
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decisions are made (Macmann, et al., 1996). This information is necessary to develop the 
most effective method to address the problem (Deno, 1995). 

Researchers have hypothesized the significance of various factors in dete rminin g the 
effectiveness of problem-solving and intervention implementation, including teacher 
acceptance of potential interventions, the integrity with which interventions are implemented, 
and intervention outcome (Elliott, 1988a; Gresham, 1989). However, little applied research 
has been carried out to confirm which critical factors apply in actual school settings. Further 
exploration of these variables is needed to understand their significance. The current 
examination begins with a look at the social validity of school-based interventions, as this 
topic subsumes other relevant variables. 

Social Validity 

Social validity is a judgment regarding whether an intervention or program has 
validity in the eyes of the society in which it is implemented. It comprises an important part 
of determining the acceptability and social usefulness of any intervention. Wolf (1978) first 
formally defined social validity as having three components: (1) The social significance of 
the goals (Are these specific goals what society wants?), (2) The social appropriateness of 
the procedures (Do participants consider the treatment procedures to be acceptable?), and (3) 
The social importance of the effects (Are consumers satisfied with the effects of the 
intervention, including any unintended effects?). Social validity can be measured at a variety 
of levels, from an individual family or classroom, to a school, local co mmuni ty, state, nation, 
or multinational level. The particular level chosen will depend on the purpose of the 
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Components of Social Validity 

Social significance of the goals includes determining the intervention target is 
necessary and appropriate, in general, as well as for this specific person. At times, goals of 
different consumers may conflict and the subsequent intervention needs to reconcile these 
differences. Another issue in the social validity of an intervention target is determining the 
standard to use for establishing the goal. This standard usually involves comparing the 
performance of the target individual to another individual who has been determined to be 
competent in the skill of interest (Van Houten, 1979). However, this may not always be the 
most appropriate standard because competency in a particular skill has not been well defined, 
or may have many definitions. At times, the usual standard may not be appropriate for this 
particular individual (the reading competency standard for a high school student with a 
significant cognitive impairment should not be the reading competency of a typical 10 th grade 
student). 

Social appropriateness of the procedures used during the intervention process has 
received a great deal of study (Gresham & Lopez, 1996; Reimers, Wacker, Cooper, & 
DeRaad, 1992). Acceptability of intervention techniques has been studied for clinical needs, 
as well as for educational needs. A limitation of some of the research concerning intervention 
procedures is many times all consumers affected by the treatment/intervention have not 
included in evaluation of its acceptability. Direct consumers are often asked about the social 
acceptability of an intervention, but more indirect consumers are not (Finney, 1991). For 
example, often clinical treatment staff are asked about acceptability of a treatment, but 
patients, family members, and community members are not included in evaluation of 
acceptability. 
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Measuring Social Validity 

Measurement of social validity also has been used to determine if the outcome of an 
intervention makes a clinically important change in the client's life. Judgments concerning 
intervention effects need to go beyond statistical significance to practical and significant 
change in a person’s life. Baer, Wolf, and Risley (1968, 1987) emphasize the importance of 
examining effectiveness as part of any evaluation of social validity. Some interventions 
include procedures designed to meet an important goal, and judged to be acceptable, but are 
not effective in meeting this goal. As a result, these interventions lack social validity in spite 
of the good intentions of planning and implementation agents. 

One method of measuring the social validity of intervention effects is using a social 
comparison criteria to measure the social validity of planned goals (Kazdin, 1977). The 
criteria are set by comparing the target individual to a socially acceptable standard (standards 
are often not objectively defined in the community). The standard used for social comparison 
must be carefully chosen, and needs to be appropriate for the individual and the intervention. 
Baer, et al. (1987) also discussed use of a cost-benefit ratio analysis to measure effectiveness 
of an intervention. If the amount of behavior change does not equal the amount of effort 
(effort can be measured in many ways, including time, cost of materials, number of people, 
etc.), then the effectiveness of the intervention is in question. A cost-benefit analysis can be 
used to help select interventions that are most effective and efficient for all concerned. 

Social validity data are often gathered by asking consumers their opinions about an 
intervention. Consumers who evaluate interventions include direct consumers (primary 
recipients of the intervention; e.g., child with a reading behavior problem and his/her 
teacher), indirect consumers (individuals strongly affected by the intervention; e.g., parent of 
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the child, or the school who hired the interventionists), and the extended community (those 
who live in the same community as the direct consumers; e.g., store clerk who waits on the 
child, people who use the same community center as the child; Schwartz & Baer, 1991). 

Including opinions of all consumers (direct, indirect, and extended) is important in 
determining an intervention's social validity, but this is often not done. If a plan is effective, 
but looks "odd" when implemented in the extended community, the plan may be 
compromised. As a result, interventionists or direct and indirect consumers may be less 
willing to implement the intervention again. Consideration of social validity also keeps 
intervention planners focused on planning changes appropriate for the broader context in 
which people live and work. An intervention acceptable to direct consumers, that is effective 
and implemented with integrity is much more likely to be considered socially valid than one 
that does not meet these criteria. 

In general, research concerning social validity has examined social acceptability of 
intervention procedures. Examination of social significance of goals and effects has received 
less attention (Gresham & Lopez, 1996). In addition, most research has looked only at 
consumer satisfaction. While this is an important component, it may provide incomplete 
information because it is an indirect, subjective measure. Consumer satisfaction is also 
dependent on consumer knowledge of the concern, alternative intervention strategies, and 
expected outcomes (Schwartz, 1991). Consumer satisfaction also only provides the opinion 
of a person or persons added to that of the intervention developer (Hawkins, 1991). More 
objective measures, such as direct observation of implementation of the program or 
intervention, are suggested to complement indirect (self-report consumer satisfaction) 
measures (Hawkins, 1991; Storey & Homer, 1991). These more objective measures can also 
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be used to determine the cost-benefit ratio of the intervention to determine if the goals will be 
likely to increase benefits and decrease costs for individuals receiving interventions as well 
as the overall community (Hawkins, 1991). Examination of actual performance involves 
collecting data about implementation integrity and effectiveness. 

Verification of social validity needs to be made for both broad programs and 
individual interventions to ensure appropriateness and effectiveness. The issue of social 
validity will continue to be an important part of program and intervention development as 
more accountability is required to meet financial and ethical standards of responsibility. 

Models of Intervention Factors 

Measuring social validity of interventions is complex as it includes measurement of a 
number of components and their interrelated effects. Several useful models have been 
developed to clarify and organize understanding of an intervention’s social validity. In an 
effort to fit theories about intervention implementation with existing empirical information, 
Witt and Elliott (1985) developed a working model of the major variables that influence 
treatment or intervention effectiveness. This model begins with intervention acceptability, 
which, in turn, affects intervention use, intervention integrity, and intervention effectiveness. 
The relationship between the elements of the model is sequential and reciprocal (Elliott, 
1988a). These authors hypothesize a break between any of the elements will affect each 
remaining element. 

Intervention use refers to actual intervention implementation. Intervention integrity is 
how closely the intervention implemented reflects the original intervention plan. 

Effectiveness of intervention refers to whether the intervention changed the behavior of 
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concern as was intended and desired. Effectiveness of the intervention, then also influences 
acceptability of the current intervention and future possible interventions because effective 
interventions are viewed as more acceptable. 

Reimers, Wacker, and Koeppl (1987) expanded the Witt and Elliott model to include 
several additional factors mentioned in the research literature. This model places 
understanding the intervention (the teacher's understanding of what the intervention includes 
and how to implement it) as the first variable. It is logical that a treatment is less likely to be 
seen as acceptable if the teacher does not understand what is to be done. 

Reimers, et al. (1987) hypothesized that poor understanding leads to low compliance 
with the intervention, which in turn, results in low effectiveness. On the other hand, if the 
interventionist has a good understanding of the intervention, then assessment can be made of 
his/her acceptability of the intervention. After acceptability is determined, compliance and 
intervention maintenance is measured. Where Witt and Elliott (1985) include treatment 
integrity in their model, Reimers, et al. (1987) instead include compliance (trying the 
intervention once) and maintenance (continuing use). Assessment is made of the 
intervention’s disruption to a family’s routine, as this may influence maintenance of the 
intervention. An evaluation of the intervention, and possible modification of the intervention, 
are included in the model. 

Both models described here offer strengths and weaknesses when used to examine 
factors influencing intervention implementation. The reciprocal nature of the Witt and Elliott 
(1985) model emphasizes the interactive nature of the elements affecting intervention 
implementation and effectiveness. However, the Reimers, et al. (1987) model includes the 
important aspects of measuring consumer understanding, and determining which factors 
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influence use and maintenance of an intervention. Both models can help guide investigation 
of the influence of treatment acceptability on integrity and student outcomes, and have been 
used by a number of researchers to do so. The remainder of this paper will examine issues 
related to intervention acceptability, integrity, and effectiveness primarily using the model 
proposed by Witt and Elliott (1985), but will include elements from the Reimers, et al. (1987) 
model also. 



Intervention Acceptability 

An important social validity aspect of any intervention plan is the acceptability of the 
intervention from the perspective of those involved. Intervention acceptability refers to the 
judgment by intervention participants or other lay persons that it is fair and appropriate (Witt 
& Elliott, 1985; Miltenberger, 1990). Teachers who do not judge a proposed intervention to 
be acceptable (workable with their curriculum, in their classroom, with their teaching style, 
etc.) are less likely to implement the intervention or to implement it as designed. Obviously, 
if an intervention is not implemented, it cannot be effective. In addition, it appears logical 
that incomplete implementation would also limit effectiveness. 

Assessment of the acceptability of interventions proposed in the problem-solving 
process must be made during problem identification and analysis, as well as during 
intervention implementation (Reimers, et al., 1987). Without information about intervention 
acceptability, the problem-solving process, and the resulting intervention are considered to be 
at higher risk of failure. However, to date, there is not enough empirical evidence to make a 
definitive determination regarding this hypothesis. 
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A large body of literature based primarily on analog studies has examined a number 
of factors that influence ratings of intervention acceptability. Analog studies have usually 
involved providing a written description of a problem to a group of individuals (most often 
college students or practicing teachers) who are then asked to evaluate the acceptability of 
proposed interventions. Only a few studies have gone beyond the analog method; as a result, 
it is difficult to determine the actual influence of the factors described in applied settings. 

Across the studies, a number of factors have been found to be influential in 
determining acceptability of interventions. These include severity of the presenting problem, 
teacher variables, theoretical perspective of the intervention along with type of intervention 
procedures used, and intervention effectiveness. Research studies ex aminin g each of these 
factors will be discussed. 

Problem Severity 

In the majority of analog studies, researchers have described severity of a presenting 
problem as a within-child variable (Elliott 1988b), not including teacher or environmental 
variables. In most cases, researchers have found as the severity of the problem increases, a 
greater range of proposed interventions are viewed as acceptable by teachers. 

In a series of experiments, Kazdin (1980a, 1980b) studied the acceptability of several 
treatment approaches for children with a range of behavior problems. Results indicated more 
positive interventions (reinforcement) were rated as more acceptable than reductive or 
aversive treatments (e.g., time out, drug therapy, and shock). All treatments were rated more 
favorably for more serious behavior problems, but the order of acceptability did not change 
(positive treatments continued to have higher ratings). The findings of these studies have 




9 r 



15 



been upheld by a number of other researchers (e.g., Martens, Witt, Elliott, & Darveaux, 

1985; Reimers, et al., 1987; Witt, Elliott, & Martens, 1984; Witt, Martens, & Elliott, 1984). 

In contrast to most research, Elliott, Turco, and Gresham (1987) found problem 
severity did not influence acceptability ratings by teachers, school psychologists, and 
students for group contingencies designed to address behavior problems. In general, all three 
groups of respondents evaluated individual contingencies as more acceptable than group 
contingencies (groups were to receive free time when two students with problems following 
classroom rules performed acceptable behavior). In a study of acceptability and 
collaboratively planned instruction, Kutskick, Gutkin, and Witt (1991) also found problem 
severity did not influence the acceptability of the various interventions. 

Elliott, Witt, Galvin, and Peterson (1984) found an interaction between problem 
severity and intervention complexity when examining their influence on acceptability. 
Teachers were asked to rate the acceptability of positive and reductive interventions for 
behavior problems. Positive procedures were rated as more acceptable for all levels of 
severity. However, these ratings interacted with those for complexity; less complex 
interventions that required less time to implement were generally viewed as more acceptable, 
unless the problem was rated as severe. Teachers may view severe problems as requiring 
more complex interventions because the problems are also more complex. 

In addition to severity, other student characteristics have also been found to influence 
intervention acceptability. Reported levels of student intelligence influenced teacher ratings 
of acceptability in a study by Martens and Meller (1989). Both a home-based reinforcement 
strategy and a school-based response cost strategy were perceived as more acceptable for 
students with average intelligence (vs. below average). The authors hypothesized the teachers 
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may have believed children with below average intelligence would have more difficulty 
understanding the intervention procedures. 

Teacher Variables 

An interesting finding of several research studies was that more experienced teachers 
rated all potential interventions as less acceptable than did less experienced teachers (Elliott, 
1988a). Hypotheses for this finding include type of training the experienced teachers had 
received (assuming changes in training over the years), or changes in expectations about 
what teachers should do (Elliott, 1988a). Perhaps these teachers have had experience with 
ineffective interventions and thus, are more critical of all proposed interventions. On the 
other hand, teacher knowledge of a particular intervention has been found to increase 
acceptability (Elliott, 1988b). It may be easier to make a decision about the acceptability of a 
particular intervention strategy if teachers know about other potential interventions, or know 
about the possibility of success with the proposed intervention. 

Experienced teachers may also have higher standards and expectations for success. 
Fuchs, Fuchs, and Phillips (1994) examined differences between teachers with higher vs. 
lower standards for success with student achievement with teachers and students in actual 
school environments. The results showed teachers with high standards had students with 
higher rates of academic achievement. However, even though the teachers made 
accommodations to meet the students’ needs, these teachers also reported not wanting 
students with disabilities in their classes. These teachers may feel students who learn more 
slowly will keep them from moving the class to the high achievement goals they had set. 
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This finding causes concern that teachers with skills necessary to provide quality assistance 
to students with significant educational needs are also those least likely to agree to do so. 

Epstein, Matson, Repp, and Helsel (1986) studied the effects of teacher status and 
student disability level on intervention acceptability. The authors asked general and special 
educators to evaluate the acceptability of several intervention strategies for children with 
mental retardation or learning disabilities. Results indicated no significant differences in 
acceptability based on either teacher status or student disability level. 

Whinnery, Fuchs, and Fuchs (1991) found different results when comparing special 
and general education teachers’ acceptance of interventions for students with mild disabilities 
who would be integrated in general education classrooms. Special education teachers 
reported a greater willingness to assist students with disabilities in the general education 
classrooms than did the general education teachers. However, general education teachers did 
report some willingness to try to assist students. When given a list of potential strategies that 
could be used to help students, the general education teachers were generally willing to use 
the strategies, even though they also reported not understanding what they would need to do 
to implement most of the strategies. Thus, general classroom teachers were willing to help 
the students in some way, even if they did not understand the specific steps involved. The 
reason their acceptability ratings for teaching students with disabilities were lower than that 
of the special education teachers may be due to a lower level of understanding of the needed 
instructional strategies than a rejection of working with students who have special needs. 

Understanding how to implement an intervention may explain the different outcomes 
obtained in these studies. Polloway, Bursuck, Jayanthi, Epstein, and Nelson (1996) reported 
results of several teacher surveys regarding the acceptability of various adaptations for 
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students using procedures familiar to teachers. They examined teacher willingness to make 
changes in homework, testing, and grading procedures for students with disabilities, and 
found teachers generally willing to make individualized changes to meet student needs in all 
three areas. These results illustrate that it may be important to look beyond teacher status, 
number of years experience, and student need. More specific information may need to be 
gathered about teacher experiences with different interventions to help shape understanding 
of teachers’ acceptability ratings of interventions. 

Other teacher related variables, such as teacher perception of effectiveness, skills, 
time, and resources needed to implement interventions influence acceptability. Witt and 
Martens (1983) found the most important factor in determining acceptability of a specific 
intervention identified by respondents (teachers) was their perception of whether the 
intervention would help the child. Other, more secondary factors, were identified that 
included possible risk to the child, amount of teacher time, and teacher skill necessary to 
implement the intervention. Skill may influence teacher understanding of the intervention, 
amount of time required to plan and carry out the intervention, the number of additional 
resources needed for implementation, and belief in their abilities to instruct students. 
Teachers who believe they can have an effect on student learning are more willing to try and 
persist in working with children (Gibson & Dembo, 1984). 

Schneider, Kerridge, and Katz (1992) found teachers rate positive intervention 
procedures that teach new skills as more acceptable than reductive procedures that involve 
reacting after the problem behavior has occurred. These authors also found interventions that 
did not require classroom changes (e.g., family therapy, social problem solving) were rated 
by classroom teachers as more favorable. This finding may be related to another factor -- 
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time required for interventions. Teachers tend to rate interventions that require more staff 
time to plan and implement as less acceptable (Witt, Elliott, & Martens, 1984; Witt, Martens, 
& Elliott, 1984). Teachers also prefer interventions that do not require additional cost or 
material resources to implement (Witt, 1986). 



Theoretical Perspective and Procedures 

Teachers rate interventions described in more practical terms as more acceptable than 
those described in theoretical terms (Witt, 1986). This finding may be related to teacher’s 
understanding of the intervention; interventions described in simple straightforward language 
may be easier to understand than those described in theoretical jargon. However, it may also 
be related to the fact that teachers can have misconceptions about certain theoretical 
perspectives (e.g., behavioral perspective) upon which interventions are based, and teachers 
tend to avoid interventions described using terminology of these perspectives (Kehle, Jenson, 
& Clark, 1992). 

Cavell, Frentz and Kelley (1986) studied teacher reactions to different perspectives by 
giving teachers case descriptions of students with behavior problems along with proposed 
treatments that were either paradoxical (counterintuitive) or nonparadoxical. Teachers rated 
the paradoxical interventions as less acceptable than even unsuccessful conventional 
interventions. When given an explanation of why the paradoxical intervention was used, 
acceptance ratings improved, but still fell into the unacceptable range. 

In another study examining how explanations about the theory behind interventions 
can influence acceptability ratings, Singh and Katz (1985) found instruction about the 
interventions’ theory and procedures (three behavioral interventions) im proved their 




U 



20 



acceptability compared to a control intervention (humanistic parenting). Undergraduate 
college students were asked to rate the four interventions, then were provided instruction 
about the behavioral interventions, then were asked to rate the interventions again. The 
behavioral interventions increased in acceptability, but the humanistic parenting control 
intervention declined in acceptability. 

Findings from a study by Kutsick, et al. (1991) expand on these results. In an analog 
study, in which the intervention planning process involved only a teacher, only a school 
psychologist, or both working together, all proposed interventions were viewed as more 
acceptable when the planning was a collaboration between both professionals. As a group, 
these studies point to the need for participants in the problem-solving process to ensure 
teachers clearly understand the proposed interventions to give them enough knowledge to 
make a determination about acceptability. Everyone involved will need to use similar jargon- 
free terminology and work collaboratively throughout the problem-solving process to make 
sure all participants understand the theoretical perspective used in the intervention. 

Effectiveness 

Kazdin (1984) hypothesized that intervention participants may view stronger 
procedures more positively because they believe these procedures are more likely to help 
achieve the established goal. In a research study examining the acceptability of several 
treatments, interventions with stronger procedures were rated as more acceptable by both 
children receiving services for behavior problems and their parents (Kazdin, 1981). In 
another study examining acceptability and effectiveness, teachers who rated an intervention 
as less acceptable also saw the intervention as less effective (Von Brock & Elliott, 1987). An 
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interaction was also found between severity of the problem and perceived intervention 
effectiveness; if the problem was only mild in severity, the acceptability was higher than for 
more severe problems. The researchers in this study hypothesized the reason for this fin din g 
was that teachers were more willing to experiment with interventions for more mild 
problems, but not for the more severe problems. It is interesting to note that this finding 
differs from those of Kazdin (1980a, 1980b) who examined severity of problems and 
acceptability of interventions. Differences in these findings may be related to different 
respondents used in the studies (Kazdin’ s early studies used college students, and several 
other studies used employed teachers). Another possible reason for the conflicting research 
results may be differences between specific problem behavior described in the studies. 

Examination of the influence of treatment effectiveness and negative side effects was 
described by Kazdin (1981). He found the described effectiveness of treatments did not 
influence acceptability ratings. However, the presence of negative side effects reduced 
acceptability, with stronger negative effects reducing acceptability more significantly. 
However, this change did not have any effect on the rank ordering of the acceptability of the 
interventions; positive interventions continued to receive higher acceptability ratings than did 
other intervention types. 

In contrast to Kazdin's (1981) findings, other studies have found effectiveness of 
treatments does affect acceptability (Witt & Elliott, 1985; Whinnery, et al., 1991). Reimers 
and Wacker (1988) conducted a study with parents of children treated in a clinic for behavior 
problems. They found initial ratings of acceptability of a treatment had less impact on later 
ratings of acceptability (after several months of intervention) than did intervention 
effectiveness. This finding was expanded with the study by Reimers, et al., (1992). These 
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authors found acceptability and intervention effectiveness were positively related to 
continued use of the intervention by parent report three and six months after beginning 
implementation. One strength of these two studies is that parents responded to actual 
interventions, rather than to analog case descriptions. Another strength is that measurement 
of acceptability was made both before and after implementation of the interventions. 

Summary 

A significant limitation of the intervention acceptability research is the reliance on 
analog studies (especially those that use college students rather than practitioners). Another 
limitation is the lack of research on more academically oriented interventions. Most research 
has focused on acceptability of interventions designed to address behavioral concerns. 
However, many interventions involve learning and/or academic concerns, and it is not known 
if the same or different factors influence acceptability of these types of interventions. 

As these studies illustrate, intervention acceptability is influenced by a number of 
factors including time, positive vs. negative procedures, side effects, effectiveness, and 
teacher experience. Lennox and Miltenberger (1990) organized a number of these factors in a 
hierarchy of order for consideration - efficacy considerations (motivation, effectiveness), 
secondary effects (side effects, abuse potential), legal and social implications (precedence, 
social acceptability, intervention restrictiveness/intrusiveness), and practical considerations 
(staff competence and cooperation, efficiency, and cost effectiveness). 
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Intervention Integrity 

Intervention acceptability is hypothesized to influence intervention integrity. 
According to this logic, interventions rated as more acceptable are more likely to be 
implemented as designed than those rated unacceptable. As illustrated by Witt and Elliott’s 
(1985) model, the integrity with which an intervention is implemented is considered to be a 
very important factor in bringing about student change. Integrity is related to both 
intervention acceptability and effectiveness. Effective interventions are hypothesized to be 
those implemented with greater integrity, and as a result those that produce more socially 
valid results. 



Importance of Integrity 

Salend (1983, 1984) described lack of intervention integrity as a potential threat to the 
internal validity of behavioral interventions. The relationship between behavior change and 
intervention implementation cannot be established without information regarding 
implementation integrity (Gresham, 1996, March). Knowledge about internal validity and 
integrity is also needed to compare the effectiveness of different interventions to determine 
the most appropriate and effective intervention for each case (Moncher & Prinz, 1991). 
Internal validity is important for the individual intervention, as well as for determination of 
external validity. 

External validity is also an important consideration in intervention integrity, as this 
form of validity will influence evaluation and replication of interventions in natural settings 
(Moncher & Prinz, 1991). Interventions may be effective in one setting and not in another, 
but without information about implementation integrity this information may be lost. Another 
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factor important in external validity is generalizability of the intervention from original 
training settings to other pertinent settings (Moncher & Prinz, 1991). An intervention may be 
seen as successful when a student who was previously unable to read starts to read material 
when working one-on-one with a teacher with a reading text, but the intervention is not truly 
successful until that student can read other materials independently (science book, magazines 
at home, etc.). Factors of internal and external validity are also important in determining the 
social validity of interventions (Wolf, 1978). 

Integrity is viewed as important in both program and individual intervention 
implementation. Carta and Greenwood (1989) describe a number of factors that influence 
integrity and illustrate the need for measuring integrity. These factors include (1) lack of 
clarity in describing the intervention, possibly resulting in various interpretations of how to 
implement; (2) implementation difficulty in terms of number of people, time, effort or 
materials needed; (3) lack of incentives for staff to implement the intervention or program as 
designed, and (4) lack of staff skills to implement procedures successfully. 

Measuring Integrity 

The importance of measuring treatment integrity for both practical and ethical reasons 
was emphasized by Shapiro (1987). It is difficult to determine the effect of a specific 
intervention in research and practice if no measurement is made of how the intervention was 
implemented (any number of outside factors may influence behavior change, and 
measurement of integrity can help control for, or explain, some of the factors). Lack of 
integrity information can result in incorrect implementation of interventions, which may lead 
to a lack of progress, and time and resources are in too short supply to waste. Shapiro (1987) 

O 

ERLC 






also points out that it is ethically unsound to attempt to change people's behavior without 
accurately describing components of the intervention, as well as their effects on persons 
involved. 

Several authors have emphasized the importance of measuring intervention integrity 
in intervention planning and evaluation (Gresham, 1989; Telzrow, 1995; Tilly & Flugum, 
1995). Yet, little effort has been directed toward examining intervention integrity in either 
research settings or during field-based intervention implementation efforts (Gresham, 1989; 
Gresham Gansle, & Noell, 1993). It is difficult to stay on course without some sort of 
compass that can be used to check the charted route. 

Peterson, et al. (1982) reviewed a number of empirical studies published between 
1968 and 1980 and found only 16% of the studies discussed measuring intervention integrity. 
An update and expansion of the Peterson, et al. (1982) study examined studies from 1980 to 
1990, and reported only 16% of intervention studies conducted in school settings provided 
information about intervention integrity (Gresham, Gansle, Noell, Cohen, & Rosenblum, 
1993). However, those studies that did measure integrity reported a positive relationship 
between high levels of implementation and intervention effectiveness. It can only be assumed 
the number of intervention evaluations that include assessment of integrity in non-research, 
applied settings is almost nonexistent. 

Valuable information to direct future research on intervention integrity is provided by 
Gresham (1989), who outlined a number of factors potentially related to intervention 
integrity. These include (1) complexity of interventions, (2) time required to implement 
interventions, (3) materials and resources required, (4) number of intervention agents, (5) 
perceived and actual effectiveness of the intervention, and (6) motivation of intervention 
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agents. It should be noted that these factors are also related to intervention acceptability. 
While each of these factors may be important, there is not currently enough empirical data to 
determine each factor’s relative influence. 

Gresham (1989) also provides guidance for further research by highlighting a number 
of technical issues involved in measuring intervention integrity. These issues include 
specification of all intervention components, degree of deviation from the treatment protocols 
and amount of behavior change, and psychometric issues involved in assessment of integrity 
(accuracy, reliability, and validity of measures). Measurement of integrity can be made via 
direct or indirect methods. 

Direct methods include observation of the intervention by a non-participant observer, 
while indirect methods can include self-monitoring and self-report, rating scales, or 
checklists (Gresham, 1989, Tilly & Flugum, 1995). Indirect methods may have the potential 
to be cost effective measures that can be used on a continual basis with interventions in 
applied settings, but not enough research currently exits to make a determination of their 
accuracy in measuring integrity. Gresham (1996, March) outlines a simple and efficient 
method useful for both direct and indirect assessment. This system involves outlining the 
specific steps in an intervention, arranging the steps in a checklist, and using the checklist to 
monitor integrity. 

While the knowledge and technology exist to begin examining intervention integrity, 
few have attempted to integrate assessment of integrity into research or practice. Those who 
have attempted to assess integrity provide evidence that implementation integrity has a 
positive influence of intervention results (Gresham, Gansle, Noell, Cohen, & Rosenblum, 
1993; Peterson & McConnell, 1996; Shear & Shapiro, 1993). However, these studies all used 
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direct observation to measure integrity, which may be too costly to use as a part of routine 
practice. 

One recent study used a more cost efficient method to assess integrity. The ongoing 
progress monitoring data collected to measure child progress was used as a reflection of 
integrity. If all the steps in the progress monitoring system were implemented, it was 
assumed all steps in the intervention plan were implemented (Allinder, 1996). This study 
illustrates cost effective methods for assessing intervention integrity may be available, and 
need to be investigated to determine if data gathered for progress monitoring accurately 
reflect intervention integrity. 

Another recent study examined the relationship between implementation integrity and 
feedback given to teachers about implementation (Noell, Witt, Gilbertson, Ranier, & 
Freeland, 1 997). This study examined how teachers implemented intervention programs 
designed as part of a consultation process. Teachers received either consultation to help plan 
the intervention or consultation during intervention planning along with feedback about their 
performance during implementation to ensure integrity. Teachers who received performance 
feedback maintained integrity of the intervention at higher levels than those who received 
only consultation. This study also used a less expensive method of gathering integrity data by 
collecting permanent products of the intervention (e.g., grade on assignment, reinforcer given 
is recorded; Noell, et al., 1997). In an examination of several studies, Witt, et al. (1997, May) 
also noted teachers initially implemented interventions with integrity, but did not continue 
with high levels of integrity unless they received some sort of prompting or feedback. These 
conditions were necessary for the entire length of the intervention; integrity dropped if 
prompting or feedback were discontinued. 
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In addition to providing feedback about the interventionists’ performance, other 
suggestions to improve intervention integrity include use of intervention manuals and 
training of interventionists, consultation, and direct supervision of interventionists 
(Gottfredson, 1993; Greenwood, Terry, Arreaga-Mayer, & Finney, 1992; Moncher & Prinz, 
1991). Improving intervention integrity is emphasized because it is hypothesized to be 
necessary for an intervention to be effective. 

Intervention Effectiveness 

As mentioned previously, intervention effectiveness is closely tied to both 
acceptability and integrity. An effective outcome is the goal of all intervention activities. The 
importance of effectiveness in the intervention implementation process can be seen in the 
emphasis on this component in the models developed by Witt and Elliott (1985) and 
Reimers, et al. (1987). Yeaton and Seachrest (1981) also describe effectiveness as one of the 
central components of interventions along with intervention strength and integrity. 

A critical issue in measuring intervention effectiveness is first defining what makes a 
particular intervention effective (Yeaton & Seachrest, 1981). In each intervention, the 
participants must determine how they will know the intervention is effective for this student. 
It may initially appear to be an easy process to simply define the goal for the student to 
achieve, but this can become very complicated. Questions arise that do not have simple 
answers. Should the student’s success be compared to other students in the class/school? 
Should a more individualized measure of success be used? Should an intervention be defined 
as effective even if the student did not reach the established goal, but has significantly 
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improved skills in the target area? Or, should the intervention be described as effective only 
if all participants agree the problem has been ameliorated? 

An important part of intervention effectiveness is the measurement system used for 
evaluation. Measurement of student progress is also an important component of the problem 
solving process. Data gathered through regular and frequent progress monitoring increases 
the likelihood of intervention effectiveness and student improvement (Lentz, Allen, & 
Ehrhardt, 1996; Shinn, Gleason, & Tindal, 1989). When data about specific student progress 
are gathered frequently, intervention changes can be made if students are not making 
adequate progress toward the goal. Teachers and students do not have time to spend on 
ineffective interventions, so it is essential they be able to quickly determine what is working 
and what is not working. 

Deno (1997) described three issues that should be addressed when establishing the 
validity of progress monitoring procedures, including looking at growth validity (measuring 
skills that reflect growth in the target area), instructional efficiency (determining what 
elements of instruction are included in the monitoring system), and instructional 
effectiveness (using the monitoring system to help the teacher improve instruction). These 
guidelines can be helpful in developing any progress monitoring system. In addition, it is 
vital that the progress monitoring data be used on a regular basis to evaluate student progress. 
Gathering data is not enough, it must actually be used to monitor and modify the intervention 
as needed. 
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Conclusion 

Teachers and schools are responding to the increasing demand for individualized 
assistance for students who are having difficulty in the general education curriculum 
(Graden, Casey, & Christenson, 1985; Graden, et al., 1988). However, the resources 
available to schools and teachers have not been increasing at the same rate as the needs. As a 
result, teachers and schools need to use their time and materials in the most efficient manner 
as possible. One way to maximize efficiency is to ensure that truly effective interventions are 
used. Effectiveness can be viewed as part of a larger picture of social validity, which also 
encompasses intervention acceptability and integrity of implementation. 

There has been notable progress in examining the individual issues related to social 
validity. However, very little research that combines these issues in applied settings with 
actual intervention programs has been undertaken. This type of research is needed to further 
examine conflicting findings from previous research, as well as explore the relationship 
between the factors of intervention acceptability, implementation integrity and intervention 
effectiveness. These variables also need to be examined in natural settings with actual 
interventions. Most available research has used only analog studies to examine the issues. 
Schools, teachers, and parents need information that can be applied to their situations to help 
individual students in need. 
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PART I. LOCAL OBSERVATION STUDY 
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INTRODUCTION 

The local observation study was undertaken to examine issues regarding intervention 
acceptability (from the teacher’s viewpoint), implementation integrity, and intervention 
effectiveness. This study was conducted with teachers serving children whose skills and/or 
behaviors were significantly discrepant from expectations and who needed specialized 
individual assistance. Schools considered to be local to Iowa State University participated in 
having actual interventions observed. 

These schools were chosen for reasons other than mere convenience; the Area 
Education Agency (AEA) serving these schools has instituted changes that influence 
intervention processes for local schools and teachers. Iowa is divided into 15 regions, each 
served by an intermediate education agency (AEA) that provides support to schools in staff 
development, media production, and special education support staff. The AEA serving the 
schools in the central Iowa area has made significant changes in the pre-referral intervention 
and assessment for special education eligibility processes, which influence how teachers 
work with children in general education classrooms (Ikeda, Tilly, Stumme, Volmer, & 
Allison, 1996). Schools engage in more formalized interventions (that are documented in 
writing) in general education classrooms, both for students who are experiencing difficulty 
with the typical school curriculum, and for those who might be considered for special 
education services. This makes these schools excellent choices for looking at factors that 
influence intervention implementation and effectiveness. 

The local observation study was designed to examine factors that influence 
acceptability of interventions, integrity of intervention implementation, and effectiveness of 
interventions in school settings. Specifically, the following questions were addressed: 
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(1) What is the relationship between teacher ratings of intervention acceptability and 
intervention integrity? 

(2) How well can teachers accurately measure intervention integrity as part of the 
intervention implementation process? 

(3) What is the relationship between intervention integrity and student outcome? 

To answer question one, data were collected using interviews and scale 

questionnaires to complete correlational analyses. Correlational data were gathered to answer 
question two using intervention checklists developed for this the study. Question three 
involved gathering descriptive and correlational data through interviews and observations. 

It was hypothesized that higher levels of acceptability will be related to higher levels 
of intervention integrity (question one). It was further hypothesized that higher levels of 
intervention integrity will be related to more positive student outcomes (question three). No 
hypotheses were developed for questions two, which was an exploratory question. 
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METHODOLOGY 



Participants 

Project participants included 10 elementary school teachers from eight local school 
districts. Teachers were recruited for participation via information sent to elementary level 
(kindergarten through sixth grade) building principals in the central Iowa area. Principals 
were asked to share information with teachers using enclosed flyers, or another method of 
their choice. Several principals included information in a staff newsletter, mentioned the 
study during staff meetings, gave the flyers to individual teachers, or posted the 
informational flyers where teachers could easily see them. Principals were called, 
approximately four to five weeks after the initial mailing of information, to remind them of 
the study and to ask for further assistance in recruiting teachers. 

Teachers who were working with students on an individualized intervention plans, 
and who were interested in participating, called the principal investigator. During this initial 
contact, the teacher was informed about the intervention components necessary for inclusion 
in the research study. Interventions needed to include the following components: (a) based on 
a written plan, (b) include direct instructional assistance (include more than only indirect 
assistance such as modifying curriculum, changing workload, contacts with parents, but 
include activities such as teaching a new skill in math, or using a reward system with the 
student that includes direct feedback), and (c) student progress on the skills addressed by the 
interventions be monitored on frequent basis. 

After determining an intervention plan included all components in the research 
study’s criteria, each teacher was sent consent forms for her own participation, as well as for 
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the students’ participation. Even though it was the teacher’s behaviors being studied, because 
these behaviors occurred in the context of working with an individual student, permission to 
allow observation of the student, as well as examination of written information collected 
about the student (written intervention plan and progress monitoring data), was sought from 
each student’s parents. Included in the packet sent for teachers to pass on to parents was a 
cover letter, a description of the research study, and a consent form requesting parents allow 
their child to participate in the study. Parents were asked to return the signed consent form to 
the principal investigator at Iowa State University. Consent forms were returned to the 
principal investigator rather than the teacher for the sake of efficiency. This method of 
obtaining permission for student participation was used in an effort to protect student 
confidentiality until parents consented to participation. Teachers and parents received a copy 
of their consent forms after the principal investigator also signed the form. 

Recruitment efforts resulted in 10 teachers who agreed to participate, and who were 
able to secure parental permission. All participating teachers were white females, with an 
average age of 38, and an average of 14 years of teaching experience. The participating 
teachers included seven general education classroom teachers in kindergarten through fourth 
grade. Three participating teachers worked as resource/at-risk teachers serving students 
receiving special education assistance (students who had IEP’s) and/or students who are 
considered to be at-risk for school failure. The Iowa Department of Education had obtained a 
waiver from the U.S. Department of Education for local schools to serve students considered 
at-risk, for a limited amount of time, in resource programs. Three teachers in this study 
served target students in this manner. The number of students served in each classroom 
varied widely as a result of the different types of classrooms observed, with between 4 and 
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30 (M=20) students in each classroom. The number of students in each classroom who had 
an IEP varied between 0 and 16, with an average of 4. Students in each classroom who 
received special assistance via written individualized plan, but who were not identified to 
receive special education ranged from 0 to 16, with an average of 3. The large variability in 
the number of students in each classroom and students with IEP’s and intervention plans can 
be explained by the different types of programs that teachers served. Several teachers served 
students in more than one type of program (e.g., Title I and ESL), and teachers described 
their students according to the types of programs in which they received services. All 
teachers worked in small school districts, seven of which were located in small towns and 
served primarily rural areas. One school district was adjacent to a large metropolitan area, but 
was not part of the urban district. 



Measures 

Data were gathered for the study using four different measures; a description of each 
instrument and its purpose is found below. The first three measures (Teacher Intervention 
Interview, Intervention Checklist, Intervention Observation Form) were developed for use in 
this study; the fourth measure consisted of individual child progress monitoring data 
collected as part of the intervention (and developed independent of the research study). 
Intervention acceptability was measured using the Intervention Comparison Scale that was 
included as part of the Teacher Intervention Interview. Intervention integrity was measured 
using the Intervention Checklist, developed from information gathered during the initial 
Teacher Interview. Child outcome data were gathered from progress monitoring records 
collected by the teachers. 
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Teacher Intervention Interview (TII) 

The Teacher Intervention Interview (TII) was developed to provide information about 
the teacher and her classroom, general experiences with and attitudes about interventions, the 
specific target intervention, and teacher perceptions regarding the acceptability of the 
interventions implemented. Information gathered as part of the interview was used to develop 
measures of intervention acceptability and integrity. Two forms of the TII, the Initial 
Interview and Closing Interview, were used to gather data See Appendix A for a copy of 
both interview forms. 

TII - Initial Interview. 

The TII - Initial Interview was used when beginning the investigation with each 
teacher. This form is divided into four sections: (1) background (demographic) information, 
(2) general questions about interventions (teacher experiences, attitudes about interventions, 
school policies and local influences on interventions), (3) questions addressing the specific 
intervention under investigation, and (4) the Intervention Comparison Scale (ICS). The ICS 
is a 21 item measure of acceptability using likert response questions (a more detailed 
description is given below). 

The third section of the TII - Initial Interview was the longest and most detailed 
section. It included questions about planning the intervention (the process and who was 
involved), materials needed, personnel involved in implementation, the actual steps involved 
in implementing the intervention, and how data about child progress were gathered. During 
the interview, a great deal of attention was given to describing the actual steps involved in 
implementing the intervention as this information was used in developing another measure - 
the Intervention Checklist (IC). Teachers were each asked to describe every step they 
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performed with the student as specifically as possible, with focus on what the teacher did and 
not necessarily on the student’s responses or behaviors. 

Til - Closing Interview. 

The TII - Closing Interview was used after the intervention was completed to check 
with the teacher to see if any changes had occurred during the course of the intervention and 
to ask questions about student progress/outcomes. This form contained only three sections: 
(1) background questions, (2) specific intervention questions (questions about materials and 
personnel needed, support provided/needed, changes in intervention steps or student needs), 
and (3) a variation of the ICS that included 17 items. This scale differs slightly from the one 
used with the Initial Interview. 

Intervention Comparison Scale (ICS). 

As mentioned previously, the ICS was designed as a measure of overall acceptability 
and has two very similar versions, one for the Initial Interview (called Initial Intervention 
Comparison Scale -- IICS) and another for the Closing Interview (called the Closing 
Intervention Comparison Scale -- CICS). See Appendix B for copies of each form. Four 
questions pertaining to intervention planning appear on the IICS but are not included on the 
CICS. In addition, wording of questions was changed to the past tense in the CICS. 

The ICS was based on several instruments used in analog studies to measure teacher 
acceptability ratings of behavioral interventions. Scales examined and used as an initial basis 
for the ICS included the Treatment Evaluation Inventory (TEI; Kazdin, 1980a), the 
Intervention Rating Profile (IRP; Witt and Elliott, 1985), and the Behavior Intervention 
Rating Scale (BIRS; VonBrock & Elliott, 1987). All these scales had been subjected to factor 
analysis and were found to be reliable measures of acceptability (Witt & Elliott, 1985). 
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None of the above scales was quite appropriate for use in this study as the questions 
addressed only behavioral interventions. For example, the BIRS included the question, “Most 
teachers would find this intervention appropriate for behavior problems in addition to the one 
described” (VonBrock & Elliott, 1987, p. 135). Items were selected from the scales that 
could be adapted to include a wider variety of interventions (i.e., academic and behavioral 
interventions), and interventions that were actually being implemented in a school setting. 

For example, one question on the ICS, “How acceptable do you find this intervention for this 
child’s needs?” was very similar to questions on the TEI and IRP. Additional items were 
selected to address areas previous research suggested influence intervention acceptability, 
such as time needed for the intervention, severity of child need, and effectiveness of the 
intervention. 

As mentioned earlier, the ICS form used in the initial interview differed slightly from 
the ICS used in the closing interview. Four items used to measure teacher involvement in 
planning the intervention, such as opportunity to choose from proposed interventions and her 
understanding of how to implement the intervention were included in the initial form, but not 
on the closing form. These questions were not included as planning activities occurred at the 
beginning of the intervention and were not applicable for the closing interview. 



Intervention Checklist (IC) 

The Intervention Checklist (IC) was used to measure the integrity of intervention 
implementation. The IC consisted of the specific steps teachers used with each child during 
the intervention, and was designed during the TII - Initial Interview (see above description of 
section three). The steps were developed to accurately reflect the intervention 
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implementation plan as well as what the teacher actually did with the student on a regular 
basis. Each step included in the IC was coded according to a likert scale: (1) did not do the 
step, (2) made some attempt to implement the step, (3) implemented the step, and (4) not 
applicable. The category “not applicable” was included because some steps were done only 
on certain days of the week or were dependent on a specific student response. For example, a 
teacher may have given a student a math drill sheet every Tuesday or may have only used a 
correction procedure when a student failed to follow stated directions. 

Data for the IC were collected using two methods: self report by teachers and direct 
observation by research assistants. Teachers were asked to complete the IC at least three 
times per week if they implemented the intervention four or five days per week, and every 
time they implemented the intervention if this occurred one to three days per week. 

Observers collected IC data each time they observed an intervention; observations were 
scheduled one or two times per week. Teachers were asked to make sure they completed the 
IC on all days that the observers also collected data. 



Intervention Observation Form (IOF) 

The IOF was used by the research assistants to collect additional data about 
contextual variables surrounding the intervention. Data were gathered about the intervention 
setting (length of intervention, number of students and adults in the room, group size). In 
addition, information about the classroom environment (rules and procedures, transitions, 
interruptions), teacher behaviors (clarity of explanations, monitoring of the classroom, 
instructional feedback, waiting for students to respond), and target student participation 
(attending, following directions, actively participating) was gathered. Each of the last three 
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categories of data were gathered using a scale of not occurring, occurring sometimes, 
occurring frequently, and unclear if the behavior occurred. A copy of this form can be found 
in Appendix C. 



Child Outcome Data 

Child outcome data were obtained from the progress monitoring data collected by 
participating teachers or support personnel as part of the intervention plan. The purpose of 
this progress monitoring was to measure change in the students’ skills addressed by the 
intervention. It is intended to be more specific than assessment of general progress in the 
classroom and/or curriculum, but not so specific that it measures only a skill that may be 
mastered in one or two training sessions. 

The progress monitoring measures were developed as part of the intervention, and 
collected independently of participation in the research study. However, teachers were 
informed participation in the study included using a written progress monitoring component 
with the intervention. Teachers were asked to make copies of any information they collected 
as part of their progress monitoring activities (e.g., charts, anecdotal notes, progress 
monitoring data sent home to parents). Teachers provided photo copies, and descriptions of 
the data they collected at the closing interview. 



Training 

The principal investigator and four research assistants collected data for the study. 
Three research assistants were graduate students, and one was a special education teacher; all 
had previous experience or were receiving training to work in elementary school settings. 




*-> 9 



42 



Data collection training was designed and implemented by the principal investigator for all 
research assistants. Training for completing the Teacher Intervention Interview (Til) forms 
consisted of discussion of each question, definitions of specific terms, and description of 
interview procedures. Next, research assistants practiced interviews in role-play situations. 
Additional role-play practice was provided to train the research assistants to develop the IC’s, 
which often required more detailed questioning of the teachers. Training for observations 
using the IC’s and the IOF also consisted of discussion of descriptions and definitions of 
each category, along with practice in role-play situations, and videotaped intervention 
examples. The videotapes included role-play and some actual classroom examples. During 
training, observers obtained an interrater reliability agreement of 100% for the IC over three 
consecutive training sessions. Interrater agreement for the IOF ranged from 63% to 94% over 
three consecutive training sessions. 



Data Collection 

Data collection took place from March to June 1997. All teacher interviews were 
conducted in person by a research assistant or the principal investigator and were audiotaped. 
Teachers were allowed to complete the fourth section of the TII - Initial Interview 
Intervention Comparison Scale after the interview and return the form to the research team at 
the next meeting if they wished. At the time of the initial interview, a copy of the written 
intervention plan was also obtained. 

Information for the IC was gathered from each TII - Initial Interview and then 
checked with the principal investigator for comprehensiveness and clarity. Each teacher then 
approved the description of the IC steps before the checklist was used. Observers provided 
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copies of the IC to each teacher, along with a set of directions describing how to complete the 
checklist. Each teacher was asked to gather information about the intervention according to 
the schedule specified in the previous section describing the IC. 

Observers gathered both IC and IOF data during each observation. Observations were 
scheduled to begin immediately before the intervention was implemented and last until it was 
completed, if possible. Some interventions were ongoing throughout the day (e.g., behavioral 
interventions focused on following directions), so teachers were asked to schedule a time 
when the observer would be most likely to see the intervention implemented. Observers 
entered the classrooms as unobtrusively as possible so as not to interrupt ongoing activities. 
Students were aware of being observed; however, every effort was made not to single out the 
target student, and teachers usually informed the children that the observer was there to see 
how she worked with students. 

After termination of the intervention, the Til - Closing Interview was conducted. 
Teachers could complete the Intervention Comparison Scale during the interview or on their 
own and mail it to the principal investigator. Photocopies of the Child Outcome Data 
(progress monitoring information including teacher notes and graphs) were also obtained 
from the teacher at this time. 




IT 



44 



RESULTS 



Description of Interventions 

Areas of need addressed by the interventions were academic for eight of the ten 
students. The academic areas covered included reading (3), prereading (letter and word 
identification for 2 kindergartners), writing (1), and math (2). The two behavioral 
interventions consisted of following directions and assignment completion. The majority of 
interventions were conducted by the child’s general classroom teacher. Several teachers had 
some assistance from a volunteer, educational associate, and/or student teacher who 
conducted the intervention occasionally or worked with other students so the teacher was free 
to work with the target student. In three cases a special teacher conducted the intervention 
separately from the classroom teacher (classroom teachers were aware of the intervention 
being implemented and information was shared between the teachers, but they were not 
directly involved in the intervention). 

Most interventions were implemented daily, one was implemented several times per 
week during a time when the academic topic area was covered, and one was implemented 
only when the student needed the assistance to meet behavioral demands of the setting. The 
grouping in which the target students received the intervention assistance included individual 
while separated from the group (five students, with one also receiving some assistance in the 
whole group setting), in small groups (three students), and during whole group work (three 
students, with one receiving some assistance individually). While the target student was 
receiving the intervention assistance, the other children in the class were usually working 
individually or in small groups on assigned work. 
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The majority of interventions had been implemented for a significant length of time 
before the teacher became involved in this study. Three had been initiated in the fall (August 
-November), three more began during the winter (December - February), and four others 
had been initiated in the spring (March - May). Only two interventions started immediately 
prior to participation in this research study. 

In most of the interventions (seven), teachers developed the interventions with a 
Building Assistance Team (BAT; also known as Teacher Assistance Team). This group of 
teachers helps the referring teacher brainstorm possible interventions, keeps written 
documentation of the plan, and provides additional guidance if needed. The remaining three 
intervention plans were developed with the AEA support team (2), and a classroom teacher. 

Written Plan 

As mentioned previously, to be included in the study, interventions needed to be 
based on a written plan. The plans used by teachers varied widely in the clarity and 
completeness of information included. Several plans included only examples of what the 
students in the special group would work on, several other plans included notes about general 
procedures, one plan included a descriptive narrative and specific steps. Five of the 
intervention plans were written on forms used by the school’s Building Assistance Team. 
Even the plans developed as part of a formal problem-solving process through the Building 
Assistance Teams varied in the specificity of description of the problem and procedures or 
steps to follow in assisting the student. An independent expert evaluator examined the written 
plans to give an objective evaluation of their clarity and quality (all personal information was 
deleted to maintain anonymity and confidentiality). The independent evaluator was an 
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advanced graduate student who had experience working with teachers and examining IEP’s 
and intervention plans. The evaluator rated how well the students’ goals could be determined 
from the written plans using a scale from one (not at all possible to determine) to six (very 
easy to determine). Student goals referred to the skills teachers and the intervention planning 
team wanted the student to achieve as a result of the intervention. The mean rating for the 
clarity of these written intervention plans was 3.50 (SD=3.00, range 1 - 6). 

Research assistants who interviewed the teachers and developed the IC’s were asked 
if it was possible to develop the intervention checklist directly from the written plan. The 
research assistants indicated that for one plan, it was possible to develop the IC directly from 
the plan, for another plan it was partially possible, and for the remaining eight it was not 
possible to develop the IC directly from the plan. This information further points out that the 
written plans contain mostly very general guidelines about what steps should be completed 
during the intervention. 



Intervention Acceptability 
Intervention Comparison Scale 

Teachers completed the Initial Intervention Comparison Scale (IICS), a measure of 
intervention acceptability, dining the initial interview. The scale contains 21 likert questions, 
with the scale range from 1 (low) to 6 (high). Five items were reverse coded, to maintain 
consistency of response format, before further analyses were completed (items recoded: 1 1 , 
12, 13, 15, 20; see Appendix B for copy of IICS). The first question on the IICS asked about 
the severity of the child’s need and was not used as part of the overall acceptability rating. 
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but as a demographic item. The average rating of the severity of the child’s need across the 
10 teachers was five on a scale from one to six. Another item was eliminated from the scale 
because the response choices for this question did not have a linear increase from a smaller to 
a greater amount as did the other questions (“How would you rate the amount of materials 
and resources needed for the intervention? likert response format was l=None, 3 to 
4=Moderate amount, and 6=Excessive amount). Factor analysis of a related questionnaire 
used for a national survey also indicated a problem with this item (see Part II. National 
Survey Study). 

The resulting initial acceptability scale was scored by summing the remaining 19 
items, with a range of possible scores from 19 to 114. Scores were summed to obtain a broad 
range of acceptability scores. The mean acceptability rating was 95 (SD=9.13) for nine 
teachers (one teacher’s data were not used because a number of items were omitted), with a 
range from 75 to 106. These scores were all above the median score on the scale 
(median=66.5). This would be an average rating of 5 on the likert scale used in the study, 
indicating a high level of acceptability for the interventions. 

Acceptability was also measured at completion of the interventions, using the Closing 
Intervention Comparison Scale (CICS), to determine if changes had occurred in teacher 
opinions. The CICS included 1 7 items from the IICS with some slight wording changes 
(some verbs changed to past tense; see description in Measures section for more detail). The 
scale was prepared for analysis in the same way as the IICS. 

Overall scores on the 15 items from the CICS were determined for nine teacher’s 
scores (one form was not returned after the study). The possible range for this scale was 15 to 
90 (median=52.5). Teacher total scores ranged from 69 to 88, with an mean rating of 82 
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(SD=7.91). A paired f-test comparing the two scales was completed to determine if teacher’s 
ratings of acceptability had changed during the course of the interventions. In order to use the 
t-test, the scores had to be placed on the same scale; each teacher’s total score was made into 
an average rating by dividing it by the number of items on each form. These average scores 
were used to complete the /-test (IICS mean=4.96, CICS mean=5.16). Even though closing 
ratings were slightly higher, there was not a statistically significant difference between 
teacher scores at the beginning and end of the intervention (/=-1.51, p=.17). 

Profile of IICS and CICS Responses 

To better examine acceptability ratings given to items, means and standard deviations 
of teacher responses to individual items from the original 21 IICS items and 17 CICS items 
(after recoding) are listed in Table 1. These responses are further illustrated in Figures 1 and 
2 . 

Figure 1 contains data about the 21 IICS items, and Figure 2 contains the same data 
for the 17 CICS items (after recoding). On each graph, the center open box indicates the 
mean, and the top and bottom whiskers indicate a +1 and a -1 standard deviation. These 
figures also illustrate teachers rated most items above the item median (3.5) for acceptability. 
The figures illustrate Resources needed was rated the lowest of any item, but as mentioned 
previously, this item had some psychometric concerns, and was not included in the final 
acceptability scale. 

While most responses were high (indicating more acceptability), teachers had a 
relatively wide range of responses for items concerning Choosing the plan from various 
options during planning on the IICS, and Child discomfort on the CICS. A wider range of 
responses were also noted for the following items on both the IICS and CICS: Practicality of 
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the intervention given amount of time needed for planning and for implementation, Lasting 
improvements made in the child’s skills, Disruptiveness to the classroom, and Number of 
people needed to implement the intervention. Teacher ratings appear to be somewhat more 
variable on the CICS than on the IICS for Practicality of the intervention given time needed 
for planning and for implementation, and Child discomfort. This indicates that teachers 
varied more in their views of acceptability for these items at the close of the interventions 
than at the beginning. 

A comparison of item means common to both the IICS and CICS was conducted to 
determine if significant changes had occurred in teacher opinions from beginning of the 
study to conclusion of the interventions. These data are presented in Table 2. As can be seen 
from the table, only two items showed a significant change from completion of the initial to 
the closing Intervention Comparison Scales. The item Importance of Need was significantly 
higher at the close of the intervention, indicating teachers were more likely to rate the 
intervention as targeting the student’s most important need at the close of the study than at 
the beginning. In addition, Side Effects was also rated higher at the close than at the 
beginning of the study, indicating teachers believed there were fewer negative side effects of 
the intervention for the student than they had previously indicated. 

Intervention Integrity 
Intervention Checklist -- Teachers 

The Intervention Checklist (IC), developed during the initial interview, served as a 
measure of intervention integrity. Each checklist was designed to be specific enough to 
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Table 1. Means and standard deviations for IICS and CICS items. 



IICS Item 


M 


SD 


CICS Item 


M 


SD 


1. Severity of Need 


4.50 


.97 


1. Severity of Need 


4.44 


.88 


2. Suitability 


5.00 


.67 


2. Suitability 


5.56 


.73 


3. Importance of Need 


4.70 


.95 


3. Importance of Need 


5.56 


.73 


4. Planning Input 


5.60 


.70 


a 






5. Planning Involvement 


5.70 


.67 


a 






6. Understanding 


5.60 


.52 


4. Understanding 


5.67 


.50 


7. Choice of Plan 


4.90 


1.45 


a 






8. Like Procedures 


5.33 


.71 


5. Like Procedures 


5.33 


.71 


9. Plan Ideas Used 


5.67 


.50 


a 






10. Acceptable 


5.40 


.70 


6. Acceptable 


5.44 


.53 


1 1 . Time Involved b 


4.22 


.97 


7. Time Involved b 


3.78 


1.09 


12. Resources Needed b 


3.56 


.72 


8. Resources Needed b 


3.78 


.97 


13. Side Effects b 


5.11 


.60 


9. Side Effects b 


5.78 


.44 


14. Effectiveness 


4.89 


.78 


10. Effectiveness 


5.22 


.67 


15. Child Discomfort b 


5.44 


1.01 


1 1 . Child Discomfort b 


5.00 


1.58 


16. Permanent 


4.11 


1.17 


12. Permanent 


4.89 


1.05 


Improvement 






Improvement 






17. Practicality — Planning 


5.00 


1.00 


13. Practicality — Planning 


5.00 


1.66 


18. Practicality — 


4.78 


1.39 


14. Practicality — 


4.78 


1.64 


Implementing 






Implementing 






1 9. Record Keeping 


5.00 


1.00 


15. Record Keeping 


5.33 


.71 


20. Disruptiveness b 


5.22 


1.30 


16. Disruptiveness ° 


5.11 


1.36 


21. People Needed 


4.78 


1.48 


17. People Needed 


5.30 


1.95 



a Item not on CICS. 
b Item reverse coded. 
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Table 2. t test of IICS and CICS items. 



Items 


Paired Differences 
M SD 


t 


Severity of Need 
IICS -1, CICS-1 


.00 


1.00 


.00 


Suitability 

IICS-2, CICS-2 


-.56 


.88 


-1.89 


Importance of Need 
IICS-3, CICS-3 


-.89 


.93 


-2.87* 


Understanding 
IICS-6, CICS-4 


-.11 


.60 


-.56 


Like Procedures 
IICS-8, CICS-5 


.00 


.76 


.00 


Acceptable 

IICS- 10, CICS-6 


.00 


.71 


.00 


Time Involved 8 
IICS-1 1, CICS-7 


.63 


1.19 


1.49 


Side Effects 8 

IICS- 13, CICS-9 


-.63 


.74 


-2.38* 


Effectiveness 

IICS- 14, CICS- 10 


-.25 


1.17 


-.61 


Child Discomfort 8 
IICS- 15, CICS-1 1 


.75 


1.75 


1.21 


Permanent Improvement 
IICS-16, CICS-12 


-.50 


1.41 


-1.00 


Practicality — Planning 
IICS- 17, CICS- 13 


-.13 


1.36 


-.26 


Practicality — Implementing 
IICS- 18, CICS- 14 


-.38 


.52 


-2.05 


Record Keeping 
IICS- 19, CICS-1 5 


-.50 


.76 


-1.87 


Disruptiveness 8 
IICS-20, CICS- 16 


.25 


1.17 


.61 


People Needed 

IICS-2 1, CICS-1 7 

“ax.. i j 


-.78 


1.39 


-1.67 



d Item reverse coded. 
* E < .05. 
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measure intervention implementation accurately, but general enough to cover the various 
activities teachers used during the intervention. Figure 3 contains examples of Intervention 
Checklists used in the study. The prereading intervention was used with a kin dergarten 
student with a target behavior of showing progress in difficult academic areas. The student 
for whom the behavioral intervention was used had a goal to increase motivation and level of 
independence in completing assignments. 

Teachers completed the IC between zero and eight times (intervention implemented 
several times each day) per week (M=2.4). Several teachers completed the checklist on a less 
frequently than had been originally requested (see schedule description in Measures section). 



Prereading intervention 

1 . Identifies alphabet letters on flashcards. 

2. Matches upper and lower case letters (using a variety of materials with letters). 

3. Reinforcing knowledge of letters - teacher names letter and student produces 
letter (e.g., traces letter in sand, chooses a flashcard with correct letter, etc.). 

4. Review letters with flashcards — identifying letters. 

Behavioral Intervention 

1 . Teacher hands the student her work folder and tells her what work she is expected 
to do within the hour. 

2. If the student does not set the timer at her desk, the teacher will prompt her to set 
the work timer. 

3. When the student’s timer goes off (every 5 min.) the teacher gives her feedback 
about the work and fills in the assignment monitoring chart. 

4. Teacher goes over points earned for the morning with the student, and records 
points earned. 



Figure 3. Examples of Intervention Checklists (IC’s). 




67 



55 



Among all teachers, across all data collection days, the self-report rate of 
implementation ranged from 0 to 100%. The overall average of completion of steps was 91% 
for all teachers in the study (the range of average completion for individual teachers was 
between 74% and 100%). 

Intervention Checklist - Observers 

Observers completed the same Intervention Checklist used by the teachers during 
each observation session. Observations were made during 68 intervention sessions, and 
occurred between zero and three times per week (M=1.06). The percent of steps recorded as 
completed for each observation ranged between 25% and 100%, with an average of 85%. 

The overall percent of steps completed for the teachers (across all observations for each 
teacher) ranged from 62% to 100%. 

Interrater Agreement 

Observers. 

Interrater reliability data were gathered between observers for 1 6 of the 68 
observations (24% of total observations). Each observer (four total observers, with principal 
investigator and three research assistants) conducted reliability assessments with each other 
observer. Agreement was determined using a point-by-point agreement ratio (Kazdin, 1982) 
to gain as accurate an assessment as possible. Agreements were calculated by adding the 
number of agreements for each step listed, divided by the number of agreements plus 
disagreements, multiplied by 100. The overall level of interrater agreement was 89%, with a 
range from 40% to 100% (10 sessions had reliability estimates of 100%). 
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Teachers and Observers. 

Agreement between teachers and observers was assessed for each observation session 
for which the teacher also completed the intervention checklist (n=42). The same point-by- 
point formula was used to determine interrater reliability between observers was utilized 
here. The overall level of agreement between teachers and observers was found to be 73% 
(range from 25% to 1 00%). 

Teacher Comments 

Teachers had also been asked about efforts made to ensure intervention integrity 
during the initial and closing interviews. Teachers indicated they did try to maintain integrity, 
but these efforts were mainly informal - talking with others about the plan and what is 
needed (two teachers) or thinking about what the student needs (three teachers), and 
reviewing lesson plans and other records (two teachers). One teacher used the checklist 
developed as part of the study to maintain integrity of implementation. 

Intervention Acceptability and Integrity 

While the sample size in this study is too small to complete formal statistical 
analyses, some comparisons of the data can be made. Overall, both acceptability and 
implementation integrity were high. A comparison of the ranking of teachers according to 
their acceptability scores on the Intervention Comparison Scale and their overall levels of 
integrity as measured by self-report and observation, showed no noticeable patterns or 
consistencies (e.g., a teacher who rated the intervention as more acceptable also ranked high 
on integrity as measured by self report and observation). However, if only self-report 
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integrity and acceptability data are examined, the teachers who had three of the four highest 
acceptability scores on the ICS were among the four teachers with the highest integrity 
scores. 



Student Outcome and Integrity 

Progress monitoring data were collected from the teachers at the conclusion of each 
intervention. As with the written intervention plans, there was a great deal of variability in 
the types of data collected and their presentation. Only one teacher used graphs to keep 
student data, one teacher did not use frequent monitoring (the teacher who was to assist with 
this part of the intervention was absent due to health problems), the other teachers used 
various types of charts to keep the data (count of the correct number of letters identified, 
number of words read correctly from a list). The clarity of the data kept on the charts varied, 
some were understandable only to the teacher using the data, and some were very clear to 
someone who examined the data without teacher interpretation. 

At the conclusion of the interventions, the observers were asked to rate (using a six 
point scale, l=low, 6=high) how well the student skill for which progress monitoring data 
were gathered reflected skills addressed during the intervention. The mean of these ratings 
across all teachers was 4. 1 (above the median of the scale, indicating moderate agreement 
between skills monitored and skills addressed in the intervention). An expert rater (an 
advanced graduate student in school psychology who had previous experience analyzing 
intervention plans) examined the progress monitoring data to evaluate how well the 
monitoring systems reflected the skills mentioned in the written plans, as well as their 
usefulness in determining student change, and whether the goals had been met. Mean ratings 
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for how well the skills mentioned in the written intervention plans reflected the skills 
measured through progress monitoring was 3.3 (in the moderate range). How well student 
progress could be determined (amount of change in student skill) had a mean rating of 2.6 
(below the median of the scale; indicating it was not easy to determine). Achievement of the 
goal had a mean rating of 1 .8 (also indicating that it was not easy to determine from the 
available data if the student goal had been met). Because it was so difficult to determine 
student growth and progress toward goals, it was not possible to examine the relationship 
between student outcome and integrity of intervention implementation using formal 
procedures. 
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DISCUSSION 

Intervention Implementation 

The interventions examined in this study shared a number of similarities. Most 
interventions addressed academic concerns, were implemented by the general classroom 
teacher over a long period of time, and involved at least some evaluation of student progress. 
The typical intervention had been developed through a team process (BAT or AEA), but 
there was little involvement of planning team members after plan development (no active 
support or evaluation). In previous studies, teachers have indicated a desire for support when 
implementing interventions (Allinder, 1996). The teachers in this study had to develop 
specific steps involved in the intervention on their own, as well as evaluate effectiveness and 
monitor implementation integrity. It can be difficult to both implement an intervention and 
determine if it is being implemented with integrity. 

The interventions also included written plans, but of varying specificity. The 
usefulness of the plans to guide the interventions may also vary as a result. While some 
teachers may prefer general ideas they can use as a rough guide to modify and adapt to then- 
own needs, others may want more direction. These teachers may not have the time or skills to 
develop more specific steps to assist students who do not learn adequately, or make good 
progress using the typical curriculum and methods. 

Intervention Acceptability 

Teacher responses to the Intervention Comparison Scales (ICS) indicated a high 
overall level of acceptance for the interventions used in this study. Teachers gave high 
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ratings for the interventions on both the initial and closing scales, with no significant change 
from the first measurement to the second, which may be expected given the fact these 
interventions were already being implemented. In fact, a number of the interventions had 
been implemented for some time prior to involvement in the research study; teachers would 
be more likely to abandon or modify interventions they did not find acceptable. 

These interventions also tended to include components previous research has found to 
lead to greater acceptance of interventions. Relevant components identified in previous 
studies, and included in this study, were a student problem severe enough to warrant 
intervention, use of positive procedures that taught skills or taught the student to self-monitor 
for improvement, time needed to implement was not excessive (i.e., more time than the 
teachers believed they had to give), use of resources to which the teachers had access, and 
few negative side effects (Miltenberger, 1990; Reimers, et al., 1987; Witt & Elliott, 1985). 

A greater range of acceptability might be obtained if assessment could have been 
made of the alternatives suggested during intervention development. Alternatives not 
selected by teachers might have had lower acceptability ratings. To help teachers and support 
teams develop the most feasible effective interventions, more information is needed about 
which factors teachers find most important in determining their acceptance of interventions 
during the planning process. The ICS developed as part of this study might be helpful in 
providing this information, but the current study does not provide information regarding its 
usefulness during intervention planning. The length of the scales might limit their usefulness 
for routine assessment in applied settings; more research is needed to determine the 
usefulness of this and other scales. 
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Intervention Integrity 

Measurement of intervention integrity was accomplished using the Intervention 
Checklists (IC’s). For the most part, it was not possible to develop the IC’s directly from the 
written intervention plans. Interviewers needed to help teachers describe the steps and 
specify the activities teachers engaged in to assist each student. Even then, the steps were 
often more general than is recommended (Gresham, 1996, March) to describe the 
intervention in a way that makes evaluation of effectiveness possible. Most teachers 
indicated completing the checklists was not intrusive or time consuming. However, most 
teachers also did not complete the IC’s as frequently as requested. 

Teachers rated themselves as completing about 90% of the steps of their 
interventions, while observers rated intervention completion at a slightly lower rate (84%). A 
number of possible explanations for this discrepancy may exist, including not defining the 
steps correctly, observers not noticing some of the teachers’ actions, and teachers 
overestimating their completion rate. In addition, difficulties in measuring the interventions 
may have influenced the findings. For example, the steps were not equal in importance, 
length of time necessary to implement, or ease of observing. Observers may have missed 
brief or unclear steps. Teachers might have overestimated the number of steps they 
completed in a desire to look good, or over generalized the activities they completed as 
fitting the steps. 

Overall, both teachers and observers saw teachers as implementing most the 
intervention steps; however, agreement as to which steps were implemented was not as 
highas overall rates. For this variable, the mean rate of interrater agreement between 
observers and teachers was moderate at 73%. A higher rate of agreement might have been 





obtained if the IC steps could have been more specifically defined. This would have resulted 
in a clearer guide for both teachers and independent observers to follow. However, this is not 
to imply the IC steps lacked applicability for use in the study. The steps were clear enough to 
get moderate agreement between teachers and observers, as well as relatively high levels of 
interobserver agreement (89%). 

One research question posed for this study was how well teachers could measure 
intervention integrity. This data indicates that teachers appear to be able to assess integrity 
with some level of accuracy; however, there is still room for improvement. Clear 
specification of intervention steps at the beginning of the intervention, as well as support for 
implementation from the intervention team members may help increase teacher accuracy. 

Teachers completed the vast majority of the steps defined in the IC’s, indicating a 
relatively high rate of intervention integrity. However, the question still remains if the IC’s 
accurately reflected the interventions as originally intended during development. No other 
persons involved in development of the interventions, including other members of the 
intervention teams, were involved in this study. Thus, it was not possible to obtain their 
views regarding the interventions’ integrity. 

Teachers in this study also had very little on-going support or assistance from the 
intervention team that had helped plan the intervention. Previous research has shown 
intervention integrity is higher when teachers receive prompting and feedback regarding their 
performance (Gottfredson, 1993; Noell, et al., 1997). Yet, the teachers in this study did not 
receive additional support or feedback to improve their performance. 

While intervention integrity has been described as an important part of intervention 
implementation, few research studies describe how it was measured or maintained (Peterson, 
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et al., 1982; Gresham, et al., 1993). It would be expected if measuring integrity is not an 
integral part of research studies, it would not be emphasized for interventions implemented 
applied settings. Thus, the lack of follow-up and feedback by the intervention teams is not 
surprising, even if it is disappointing. However, given the need for effective and efficient 
interventions for the benefit of both students and teachers, this result needs to change. One 
way to accomplish this would be for the intervention teams to provide performance feedback 
and support to teachers implementing interventions. 

It was hypothesized that interventions with higher levels of acceptability would be 
associated with higher levels of intervention integrity. This hypothesis was not supported by 
the data obtained. All interventions had relatively high levels of acceptability and integrity; 
as a result, the lack of variability made comparisons difficult to make. Other research 
methods may need to be used to study this relationship, such as prescribed interventions that 
teachers are assigned (and cannot choose those they find acceptable). 

Student Outcome 

It was very difficult to determine student outcome and intervention effectiveness from 
the progress monitoring data. Teachers tended to use monitoring systems that were very 
individualized for their own purposes (often for short-term instructional planning) but did not 
use systems that communicated progress clearly to other persons. In addition, these progress 
monitoring systems often were used to measure short-term gains, instead of longer-term 
student improvement related to the students’ intervention goals. As a result, it was difficult to 
determine if student’s had made progress toward, or met, their goals. Most teachers indicated 
they believed the students had made progress, but they also thought the intervention (or 
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something similar) should continue the next school year, so the assumption could be made 
that the problems requiring intervention were not completely ameliorated. However, the 
degree of progress and continuing need could not be determined accurately. 

If the progress monitoring data cannot be used to determine attainment of the 
intervention goal, then decisions most probably will be made based only on teacher opinion. 
Opinion can be swayed by extraneous factors such as other student behaviors that are 
pleasing or annoying to teachers, or temporary changes in growth (e.g., the student had a 
very bad day right before the teacher reported on progress, and the teacher became concerned 
that progress was now going to stop). While teachers are often very capable of evaluating 
general student growth, their evaluations are not as precise as the information obtained from 
specific intervention progress monitoring procedures. 

Research has also indicated higher levels of student achievement can be obtained 
when frequent measurement of progress is used (Shinn, et al., 1989). The use of highly 
structured and empirically validated progress monitoring procedures could help teachers 
made appropriate instructional decisions to improve student achievement and communication 
with others (Deno, 1997). Yet, most teachers in this study did not use such procedures. 
Greater direction and follow-up from the intervention planning teams could have assisted the 
teachers in developing more useful progress monitoring systems. The guidelines described by 
Deno (1997) illustrate how the planning teams could have improved the progress monitoring 
approaches used by teachers. The teams could have ensured the procedures had instructional 
efficiency (include important instructional elements), growth validity (measure skills that 
reflect the target area), and instructional effectiveness (serve to improve instruction). 
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Given the lack of empirical data about student change, the effectiveness of the 
interventions studied cannot be determined. As a result, the relationship between integrity 
and student outcome cannot be established for these interventions. Thus, this study provides 
only limited knowledge concerning the relationship between intervention integrity and 
effectiveness. 



Limitations 

While this study provides interesting data about interventions as they were applied in 
actual settings, it also has several limitations. The study includes a very small sample of 
teachers who were all volunteers drawn from a limited geographic area. All interventions 
were observed near the end of the school year which may have affected the type of 
interventions being implemented. These interventions may be very different from those 
implemented earlier in the school year, especially those that were implemented for a short 
amount of time (and were discontinued because the goal was met, or the student showed lack 
of progress). The type of problems addressed, or the level of students’ needs may have been 
different for interventions implemented earlier in the school year. This study did not examine 
all possible interventions for their acceptability; it only examined acceptability of 
interventions that had already been selected to be implemented. Thus, intervention integrity 
may also have been affected positively. More research is needed to confirm and expand the 
results found here. Additional research could examine acceptability of interventions during 
the planning stage, as well as at the end of implementation. Research is also needed to 
determine the impact of intervention integrity on student outcome. Also, larger samples of 
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teachers implementing interventions in more schools are needed to examine relationships 
between intervention acceptability, integrity, and effectiveness. 
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PART II. NATIONAL SURVEY STUDY 
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INTRODUCTION 

The national survey study was undertaken to further explore issues raised as part of 
the Local Observation Study. Specifically, the national survey was designed as an initial 
exploration of teachers’ perceptions regarding factors that influence intervention 
implementation. The purpose was to determine if factors found to be important to 
intervention acceptability and integrity in central Iowa were also significant in other areas. 
The national study was conducted after the local observational study had been initiated to use 
feedback gathered from instrument development, as well as information gathered during the 
study to shape questionnaire design. 

A national sample was believed to be important to get a broader perspective regarding 
variables that influence intervention development and implementation. Many local areas are 
working to improve outcomes for students, but these efforts can look very different. As 
mentioned previously, Iowa is working to reform special education practices, and the local 
Area Education Agency (AEA) has made significant changes in practices and expectations 
for intervention implementation. For example, teachers are expected to provide 
documentation of intervention efforts that include frequent progress monitoring of student 
change before a student will be considered for special education services. AEA support staff 
are expected to provide assistance to teachers in developing individualized interventions and 
monitoring student progress. Other AEA’s in Iowa have adopted different methods to reform 
special education and practices with students considered to be at-risk, and other states have 
also varied in how they have chosen to focus on making changes in intervention 
implementation. The national survey was used to explore some of these differences in 
practices. 
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This study was designed to examine intervention acceptability and integrity from a 
broad range of teachers working in different states and geographic regions. Specifically, the 
following questions were addressed: 

(1) What is the relationship between teacher ratings of intervention acceptability and 
intervention integrity? 

(2) What factors influence in intervention implementation across different states? 

(3) How are intervention acceptability and integrity measured and evaluated in actual 
practice? 

To answer question one, data were collected using a questionnaire to complete 
correlational analyses. Descriptive analyses of interview and questionnaire data were utilized 
to answer questions two and three. It was hypothesized that higher levels of acceptability will 
be related to higher levels of intervention integrity (question 1). No hypotheses were 
developed for questions two and three, as these are more exploratory questions. 
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METHODOLOGY 

Participants 

Elementary school teachers from 1 1 states were selected to participate in this survey 
study. States were selected for the study both randomly and selectively. Iowa was selected 
for inclusion to serve as a comparison for the Local Observation Study, and the 10 other 
states were selected from a geographically representative random sample (see Data 
Collection section for details). Background characteristics of the respondents can be found in 
Table 3 where data are summarized for the total sample, Iowa, and the 10 states from the 
national sample. As this table shows, the overwhelming majority of respondents were female 
general education teachers in lower elementary grades (kindergarten through third grade). 
Descriptions of their classrooms are presented in Table 4. 

Measures 

The Teacher Intervention Questionnaire (TIQ; Luze, 1996) developed for this study 
was adapted from the Teacher Intervention Interview (TII) used in the Local Observation 
Study (see Part I). The TIQ consisted of four parts. The first section contained background 
questions including questions regarding gender, age, position, grades taught, number of 
students, size of community, etc. The second section asked teachers general questions about 
their experiences working with students who need specific individualized interventions or 
assistance. Interventions were defined as specific activities the teacher develops and/or uses 
to help a student with a particular difficulty. Questions asked about specific areas in which 
the teacher has assisted students, any assistance teachers received to help the students, 
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Table 3. Respondent background information. 





Total 

Sample 

N 

(350) 


% 


Iowa 

Sample 

N 

(201) 


% 


National 

Sample 

N 

(149) 


% 


Gender 


Female 


337 


97 


196 


97 


141 


95 


Male 


12 


3 


5 


3 


7 


5 


Age 


20-29 


38 


11 


21 


10 


17 


12 


30-39 


78 


22 


45 


22 


33 


22 


40-49 


138 


40 


76 


38 


62 


42 


50-59 


78 


22 


48 


24 


30 


20 


60+ 


17 


5 


11 


6 


6 


4 


Number of Years Teaching 


1-4 


36 


10 


18 


9 


18 


12 


5-9 


45 


13 


29 


14 


16 


11 


10-14 


71 


20 


35 


17 


36 


24 


15-19 


68 


20 


39 


20 


29 


20 


20-24 


54 


15 


30 


15 


24 


16 


25-29 


36 


13 


31 


15 


15 


10 


30+ 


30 


9 


19 


10 


11 


8 


Position 


General Education 


330 


94 


194 


97 


136 


91 


Special Education 


5 


1 


0 


0 


5 


3 


Special program 


5 


1 


2 


1 


3 


2 


General and Special 


6 


2 


5 


3 


1 


1 


Education 


Administration 


1 


>1 


0 


0 


1 


1 


Other 


3 


1 


0 


0 


3 


2 


Grade Teach 


kindergarten 


68 


19 


36 


18 


32 


22 


1 


72 


21 


39 


19 


33 


22 


2 


86 


25 


59 


29 


27 


18 


3 


74 


21 


43 


21 


31 


21 


4 & 5 


6 


2 


2 


1 


4 


3 


Multiage 


37 


11 


17 


9 


20 


13 


Other 


5 


2 


2 


3 


1 


1 
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Table 4. Descriptions of classrooms. 



Total Iowa National 

Sample Sample Sample 





N 


% N 


% N 


% 




(350) 


(201) 


(149) 





Number of Children in 
Classroom 



9-14 


25 


7 


10 


5 


15 


10 


15-19 


94 


27 


58 


29 


36 


23 


20-24 


166 


47 


90 


45 


76 


51 


25-29 


55 


16 


38 


19 


17 


11 


30+ 

Number of Children in 
Classroom with IEP’s 


10 


3 


5 


2 


5 


4 


0 


83 


24 


52 


26 


31 


21 


1 


82 


24 


48 


25 


34 


23 


2 


52 


15 


31 


15 


21 


14 


3 


40 


12 


22 


11 


18 


12 


4 


43 


13 


22 


11 


21 


14 


5 or more 


46 


13 


23 


12 


22 


15 



including what type and who provided this assistance, and any specific 
requirements/expectations for intervention implementation in their local schools. 

Parts three and four of the TIQ were related, asking questions about experiences 
providing interventions) for a particular student. In part three, each teacher selected a 
specific student with whom he/she had conducted intervention(s) during the past two years. 
The selected student should have had learning and/or behavioral needs that could not be met 
through the typical curriculum, and who was not receiving special education assistance. The 
teacher answered the questions regarding work with only that particular student, and did not 
include experiences he/she has had working with other students. The questions asked about 
areas addressed by the intervention, the plan used to guide the intervention, how the plan was 
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developed, who implemented the intervention, how student progress was monitored, and if 
intervention integrity had been monitored and/or maintained. 

The fourth section, labeled Intervention Rating (IR), included 12 likert questions to 
examine intervention acceptability. The Intervention Rating scale was developed from items 
in the Local Observation Study’s Intervention Comparison Scale (ICS). Seven of the IR scale 
items match items from the ICS exactly or with only minor wording changes. The other scale 
items were adapted from open-ended questions asked in the Til Initial and Closing 
interviews (for more detail, see Part I. Local Observation Study). Questions on the used a six 
point likert scale, very similar to that used for the ICS. At the end of the questionnaire, 
teachers were given an opportunity to make additional comments if they wished to do so. See 
Appendix D for a copy of the TIQ. 



Data Collection 

Surveys were sent to teachers in 1 1 states - Iowa and 10 additional states. These 10 
states were randomly selected to represent five geographic regions of the country, northeast, 
southeast, midwest, northwest, and southwest. A randomized list of all 50 states was used to 
select two states from each region. When a state was selected, the state’s Department of 
Education was contacted to obtain a random list of individual teacher names and school 
addresses for teachers in kindergarten through third grade. If the state’s Department of 
Education was unable to provide a list of individual teachers, the next state from the 
randomized list was contacted. This process continued until two states from each region were 
able to provide the needed list. School district lists were not used to maintain consistency of 
sampling procedures. Selecting teachers from a list of school districts could have resulted in 
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a greater chance of under-sampling larger school districts and over-sampling smaller ones. 
The ten states included in the national sample were: Maine and West Virginia (northeast), 
Arkansas and Mississippi (southeast), Illinois and Missouri (midwest), Idaho and Wyoming 
(northwest), and Oklahoma and Utah (southwest). 

Thirty teachers from each state were contacted and asked to complete the survey, 
resulting in a total of 300 surveys sent to the 10 states. It was not the intention of this study to 
compare data between these states, so no effort was made to obtain enough surveys to 
complete a state-by-state analysis. A disproportionately larger number of surveys were sent 
to teachers in Iowa (300 total) to compare findings with the Local Observation Study. As a 
result, a grand total of 600 surveys were sent to teachers in 1 1 states. 

Each teacher was sent a letter with the survey describing the study and asking the 
individual teacher to respond. In addition to the letter and survey, each mailing also included 
a postage paid return envelope and an individual-serving tea bag. The tea bag was included to 
provide teachers a small token of appreciation for taking time to complete the survey. The 
initial mailing to teachers was completed over a two week period during mid- April. After 
two weeks, teachers who had not returned the survey were sent a reminder postcard. Two 
weeks later (four weeks after the initial mailing), each nonrespondent was sent a second letter 
of request, a survey, and a return envelope. A total of 358 survey responses were received, of 
these 350 were usable (eight were unusable because the individuals did not complete them, 
citing as reasons that they did not wish to be included in the study or did not believe it 
applied to their current work position). This resulted in an overall return rate for the surveys 
of 58%. In Iowa, the return rate was 68%, and the return rate for all other states combined 
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was 48% (no single state had a return rate lower than 33%). All returned surveys were 
received by mid- June, 1 997. 
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RESULTS 



Intervention Acceptability 

Intervention acceptability was measured using the Intervention Rating (IR) scale 
portion of the questionnaire. Teachers answered 12 questions using a likert response format. 
The scale was submitted to factor analysis to investigate how well it was measuring 
intervention acceptability (see Appendix E for a description and results of the factor 
analysis). This analysis resulted in an 1 1 - item scale that appeared to contain several related 
components of acceptability. Scores from the 1 1 items were summed for each respondent to 
create a total acceptability score. These scores were then used in further analyses to examine 
the relationships between acceptability and other intervention factors. 

The resulting ratings of intervention acceptability tended to be in the moderately to 
very acceptable range based on the likert scale. Scores for each item were summed to form a 
total acceptability score for each respondent. Total acceptability scores ranged from 21 to 66 
(possible range from 1 1 to 66), with the mean score of 49. The mean score falls slightly 
above the moderately acceptable range (median on the scale was 38.5). 

Intervention Implementation 

This section begins with a description of intervention implementation results for all 
respondents. Following the descriptive data, a statistical comparison of the findings between 
Iowa and the national sample is made. 

Teacher responses to survey items indicated that 92% of the schools expect teachers 
to work on individual interventions for students having difficulty. However, a much smaller 
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number (n=168, 49%) indicated that there are formal guidelines for what these interventions 
should include. A small number of teachers (n=52, 15%) indicated that they did not know if 
the school had formal guidelines. Of those teachers who had formal guidelines to follow, the 
types of guidelines were most likely to involve a Teacher Assistance Team (TAT) process 
(23%). An additional five percent of respondents listed specific steps in a problem solving 
process, but did not mention that it occurs only within the TAT type process. The second 
most frequently cited formal guideline included mention of a combination of procedures 
(such as TAT process, paperwork and documentation of efforts). Other formal guidelines 
mentioned by teachers included need to document efforts, use of a problem-solving process, 
and special education referral procedures. 

Teachers were then asked if they had worked with a student who was having 
difficulty (who was not receiving special education services) using an individualized 
intervention in the past two years. The student was to have had learning and/or behavioral 
needs that could not be met through the typical curriculum. The vast majority of respondents 
(n=304, 87%) reported having worked on such interventions during the past two years. 

Teachers were asked to indicate the subject area(s) addressed by the interventions, 
and a number of teachers indicated working with the student in more than one subject area. 
The most frequently listed area was behavior problem (n=204), followed by reading (n=196), 
written language (n=152), math (n=146), study skills (n=89), and a number of other areas 
listed much less frequently (such as science n=13, spelling n=l 1, speech and language n=8). 
Types of changes made by teachers as part of the intervention included c hangin g 
instructional methods (n=240), using an incentive (reinforcement) system (n=210), c han ging 
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instructional grouping (n=196), changing the physical environment (n=176), and changing 
instructional materials (n=167). 

As teachers worked with their students on these interventions, most reported working 
with a specific plan that included specific steps to meet the student’s needs (n=254, 84%). A 
smaller number of teachers actually documented the plan in writing (n=172, 57%). Most of 
the teachers developed the plan in conjunction with at least one additional person (parent, 
educational aide, other teacher, TAT, special education support personnel). Only 50 teachers 
(16%) indicated they developed the plan alone (27 in Iowa, and 23 teachers from the 10 state 
sample). 

A significant proportion of teachers indicated that they gathered information about 
student progress/change during the intervention (n=272, 89%). The majority of classroom 
teachers gathered the progress monitoring data themselves, with a small number indicating 
they received some assistance from a teacher associate in gathering these data. Most teachers 
also reported making changes in the intervention as a result of student progress that was 
better or worse than expected (n=236, 78%). 

When asked if efforts were made to ensure the intervention was implemented as 
originally planned (intervention integrity), a large number of teachers (n=258, 85%) 
indicated that such efforts had been made. Most teachers indicated that they did this alone, 
without help from another person. When asked how intervention integrity had been ensured 
(teachers could select more than one response), the most frequently cited activities were use 
of progress monitoring data (n=156), consultation with someone (n=79), keeping a calendar 
to indicate days the intervention was implemented (n=61), using a checklist of the steps 
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(n=59), and being observed by someone (n=25). Forty-four teachers who indicated they had 
worked to ensure intervention integrity kept no formal documentation. 

The Chi-square test was used to examine possible differences between responses 
from teachers in Iowa and the national sample. Table 5 illustrates findings of these tests, with 
the only significant differences being found in the number of teachers who implemented 
individualized interventions in the past two years, written documentation of these 
intervention plans, and efforts made to ensure intervention implementation integrity. The 



Table 5. Comparisons of teachers in Iowa and national samples on intervention guidelines, 
use, and documentation. 



Variables 


Iowa 

sample 

N 


National 

sample 

N 




School expects teachers to work on 
individual interventions 


184 


136 


.00 


Formal guidelines exist for what 
interventions should include 


95 


72 


.02 


Teacher has worked with a student on an 
individual intervention plan in the past 
two years 


185 


119 


11.27*** 


Teacher used a plan with specific steps to 
work with the student 


158 


96 


1.10 


Teacher’s plan was documented in writing 


114 


58 


4.06* 


Information gathered about student 
progress/change 


162 


110 


1.54 


Efforts were made to ensure intervention 
integrity 


152 


106 


4.10* 



*E < .05. ***p < .001. 
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difference between the two groups was highly significant for the number of teachers who had 
implemented individualized interventions. Iowa teachers were more likely to report having 
implemented individual interventions with students having difficulty and using a written plan 
to guide the intervention, but teachers from the other states were more likely to report having 
made an effort to maintain intervention integrity. 

Regression analyses were conducted to explore the relationships between intervention 
integrity and a number of other variables. Examination was made of several variables that 
might influence intervention integrity, including using a written plan to guide the 
intervention, monitoring child progress, and acceptability of the intervention. Acceptability 
of the intervention was measured using the sum of the 11 item Intervention Rating scale 
discussed above. Scores from this scale were regressed with responses concerning efforts had 
been made to maintain intervention integrity. First the relationships between using a written 
plan to guide intervention implementation and acceptability were examined (see Table 6). 
Results indicated that there was not a statistically significant relationship between having a 
written plan and acceptability of the intervention; however, there was a statistically 
significant relationship between the use of progress monitoring and acceptability of the 
intervention. 

The second set of analyses consisted of examining variables that might infl uence 
intervention integrity. Included here were use of a written plan, use of monitoring of child 
progress, and acceptability of the intervention. Results of these analyses can also be found in 
Table 6. As can be seen from the data, statistically significant relationships existed between 
efforts to maintain intervention integrity and all of the variables examined — using a written 
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plan to guide the intervention, measuring child progress, and acceptability of the 
intervention. 



Table 6. Summary of regression analyses related to intervention acceptability and integrity. 



Variables 


R 


R' 


B value 


Beta 


F 


Acceptability 












Written Plan 


.10 


.01 


-1.89 


-.10 


2.38 


Progress Monitoring 


.13 


.02 


-3.11 


-.13 


5.08* 


Integrity 












Acceptability 


.13 


.02 


-.01 


-.13 


4.97* 


Written Plan 


.19 


.04 


.22 


.19 


8.64*** 


Progress Monitoring 


.96 


.93 


.97 


.96 


4579.20*** 


*P<.05. ***£<.001. 
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DISCUSSION 

Intervention Implementation 

This survey gathered information from teachers serving early elementary classrooms 
across the country. The vast majority of these teachers reported serving a student with an 
individual intervention during the past two years. Few teachers had formal guidelines to 
follow from their schools when planning and implementing the interventions. When they did 
have guidelines, the guidelines usually involved a teacher assistance team process. The 
quality of team assistance was not measured in this study, but could be expected to vary a 
great deal just as the skills of individual teachers and support services personnel vary. 

The students served via the interventions tended to have needs in more than one area, 
such as behavior and reading or science and math. The most frequently cited single area of 
student need was that of behavior problems, followed by reading. To meet student needs, 
teachers tended to make several types of changes to meet the students’ needs, with the most 
frequently mentioned change being altering instructional methods. 

Most teachers worked with at least one other person to develop the plan and included 
specific steps in the plan, but were less likely to document the plan in writing. In Iowa, 
teachers were more likely to report working with students on individual interventions and 
using a formal written plan to guide the intervention than teachers in other states. Possible 
explanations for this finding include that teachers from Iowa actually imp lemented more 
interventions (and used written plans more often), or that Iowa teachers used different 
definitions for interventions and written plans than teachers in other states (and as a result 
reported a greater number of interventions). If the first explanation is accurate, teachers from 
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Iowa may have reported implementing more interventions because of recent changes in 
special education rules and procedures (Ikeda, et al., 1996), which have increased the 
expectation that teachers will use individualized interventions. 

In general, teachers also reported keeping progress monitoring data about the 
intervention, and using these data to make intervention changes. A large number of teachers 
reported making changes in the intervention as a result of information gathered through 
progress monitoring. However, these results need to be accepted with caution. Teachers in 
the local observation study also reported gathering progress monitoring data and using it for 
decision making, but the clarity and usefulness of these data for measuring long-term student 
growth varied greatly. 

Many teachers who indicated making an effort to maintain integrity, also reported 
using progress monitoring data as their measure of integrity. Allinder (1996) also used 
progress monitoring information and found it to accurately reflect intervention integrity. This 
is a less expensive system than using direct observation, and might be a useful method of 
measuring integrity for classroom-based interventions. While the use of progress monitoring 
data shows promise, it should be used prudently given the caution previously discussed. 



Intervention Acceptability 

The Intervention Rating scale was used to gather information about teachers’ 
perceptions of intervention acceptability. This scale was submitted to factor analysis to 
examine how the items fit together. This analysis found three related components to > 
acceptability - student related concerns, practical teacher concerns and factors surrounding of 
the intervention. 




£6 



84 



Most teachers gave moderately high to high ratings of acceptability for the 
interventions they reported implementing. This finding is logical given the fact that the 
teachers were describing an intervention that they had implemented in the past and would 
probably not have continued an intervention they did not find acceptable. Teachers would 
also be less likely to describe in a survey an intervention they had not found to be acceptable. 

The fact that few teachers developed the intervention plan in isolation is important to 
keep in mind when considering acceptability of interventions. Other participants in the 
problem-solving and intervention planning process should assist in determining acceptability 
of proposed interventions, along with teachers, throughout the different intervention phases. 
At each stage of planning, implementation, and evaluation, acceptability should be re- 
assessed to look for possible changes in the teachers’ perceptions. Teachers’ participation in 
decision making can influence their willingness to implement interventions for students 
(Fuchs, Fuchs, & Bishop, 1992). When their opinions and input are sought during planning 
and selection of the intervention, teachers might view the choices more positively, or be able 
to have their ideas about acceptable interventions be included. Including teachers’ 
perceptions in all stages should help improve intervention integrity and effectiveness as well. 



ERIC 



Intervention Integrity 

Most teachers working on interventions reported that they had no formal guidelines to 
follow when planning and implementing interventions with students. This can also lead to 
interventions of varying quality, implemented with varying levels of integrity. It is important 
to know an intervention is implemented with integrity to be able to attribute behavior change 
to the intervention rather than to other changes that may have occurred (Gresham, 1996, 
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March; Tilly & Flugum, 1995). If schools would provide such guidelines and the necessary 
training to use them correctly, teachers might be better able to maintain intervention integrity 
and assist students more effectively. 

Most teachers in the survey reported making efforts to maintain intervention integrity, 
and these efforts tended to be more formal (keeping progress monitoring data, having 
someone observe them). Teachers from the 10 states included in the national sample reported 
making more effort to maintain integrity than did Iowa teachers. However, teachers from 
these states were also less likely to use a written plan. It is extremely difficult to maintain 
integrity when a plan is not written. Without a written guide, a teacher’s interpretation of the 
intervention steps could change over time. In general, the teachers reported working alone in 
maintaining intervention integrity, which is hard to do well. It is difficult to maintain an 
objective perspective, as well as be able to notice if all steps have been implemented when 
trying to manage the intervention and any other classroom demands. 

Limitations 

While this study gives interesting initial information about intervention 
implementation in a variety of classrooms, it does have several limitations that should be 
noted. One significant limitation of this study is that the data are only self-report from 
teachers who completed and returned the survey. While a majority of the surveys were 
returned, teachers who returned them might differ from those who chose not to return the 
survey. In addition, teachers may have worked in schools not representative of what most 
schools and teachers are doing to assist students having difficulty. Teachers may have 
interpreted questions about what constitutes an intervention, as well as progress monitoring 
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and intervention integrity differently from intended. They may have also provided responses 
that they believe to be socially desirable rather than reflecting actual practice, or not 
remembered the interventions accurately when considering the questions (some of the 
interventions may have been implemented up to two years ago). In spite of these limitations, 
this survey provides a starting point to learn what is going on in schools for teachers who are 
working on interventions plans with students in need. 
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GENERAL CONCLUSION 
Comparing the Studies 

Despite using different methods and participants, these two studies both examined 
issues related to intervention acceptability, integrity, and effectiveness for intervention plans 
implemented by teachers in school settings. Findings from the two studies were similar in 
some aspects, but very different in others. Most of the interventions in the local observation 
study addressed academic concerns, while teachers in the national survey more frequently 
described working on behavior problems. 

When planning interventions, teachers in both studies reported they had assistance 
from at least one other person. Assistance providers included educational assistants, other 
teachers, administrators, and parents. In the local observation study, a teacher assistance or 
child study team usually provided planning assistance. The assistance tended to end after the 
intervention had been planned; teachers in both studies reported implementing the 
interventions on their own. They also did not receive assistance in gathering and evaluating 
progress monitoring data or in maintaining intervention integrity. 

Two studies illustrated differences in the resulting intervention plans. Teachers in the 
observation study used written plans, but the steps documented in the plans were general and 
needed to be further specified to be measured in the current research study. Survey 
respondents reported using interventions plans with specific steps, but often without a written 
plan. However, teachers from Iowa were more likely to report using written plans than 
teachers from other states. This raises questions regarding the specificity of the steps used by 





88 



teachers responding to the survey. What may appear to be specific to them may be very 
vague to an outsider who would observe the teacher working with the target student. 

There was a great deal of overlap in questions used to address intervention 
acceptability between the two studies. In general, teachers responded that the interventions 
had high levels of acceptability on factors previous research had indicated influenced 
acceptability. These factors include severity of the child’s need, time needed to implement, 
number of people needed to implement, and effectiveness for the child’s need (Elliott, 1988a; 
Lennox & Miltenberger, 1 990). 

Teachers in both studies reported using monitoring systems to gather information 
about student progress. However, teachers in the local study gathered data that differed 
widely in clarity and usefulness in communicating with others interested in the intervention 
(e.g., assistance team members and parents). Teachers in the survey were not asked specific 
questions about the progress monitoring system used, but wide variations in clarity would 
also be expected. 

The purpose of progress monitoring is to measure student growth and evaluate the 
effects of interventions (Shinn, 1995). Systematic monitoring of student progress is also 
associated with greater gains in achievement (Shinn, et al., 1989). If good decisions are to be 
based on available data, then good data need to be gathered. Often teachers have not received 
training to develop and use progress monitoring systems for individualized interventions. To 
expect them to develop quality systems on their own is not realistic. Teachers need assistance 
and training to develop monitoring systems that are socially valid and useful for measuring 
student growth. This requires ongoing involvement and support from other intervention team 
members (e.g., trained administrators, teacher assistance team members, or special education 
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support staff - school psychologists, consulting teachers, etc.). Good quality progress 
monitoring may also help address the issue of intervention integrity, as Allinder’s (1996) 
study illustrated. 

Efforts to maintain intervention integrity differed between the two studies also. 
Teachers in the observation study indicated using only informal efforts to maintain integrity. 
Most of the teachers in the survey study reported making efforts to maintain integrity of the 
intervention, and usually using formal methods to accomplish this. In both studies, few 
teachers reported receiving help to maintain intervention integrity, most were left to do this 
on their own. Teachers working alone do not have access to methods found to be effective in 
increasing implementation integrity, primarily performance feedback (Noell, et al., 1997). 
Without direct feedback about implementation, teachers are left to guess if they are 
implementing the plan as the intervention team had originally conceptualized it. These data 
point to a need for more emphasis to be placed on monitoring and enhancing intervention 
integrity in school settings. If teachers could receive more direction from the person(s) who 
help plan the intervention, they might be better able to implement interventions and progress 
monitoring systems with integrity. The result could be more effective and efficient 
interventions. 



Implications 

The effectiveness of interventions rests on the acceptability and integrity of 
implementation. As schools and teachers implement more interventions for students having 
difficulty in school they do not want to spend time and effort on ineffective interventions. 
However, most schools have not taken the necessary steps to ensure effective interventions 
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are being implemented. As results of these two studies illustrate, there are several steps that 
could be taken to help teachers and schools work more effectively. These steps could be 
considered to fall under the conceptual umbrella of support. 

This support would include schools developing formal guidelines to inform teachers 
about what is needed to implement effective interventions. Teachers would then need to be 
trained about the components of these guidelines. In addition, the teachers need to have more 

ongoing help from their intervention planning teams. These teams need to change their 

\ 

procedures from assisting only with planning interventions to helping with intervention 
implementation and evaluation. The teams could help develop written intervention plans that 
include specific steps for implementation. This would help teachers know exactly what they 
need to do and improve communication with the team (who would then know precisely what 
the teacher is working on when they are providing follow-up assistance). The team could 
train teachers in how to develop useful progress monitoring systems, and provide feedback 
and support about use of the progress monitoring system as well as the specific intervention 
steps. These supports could include the type of consultation support and performance 
feedback provided in the Noell, et al. (1997) study. 

In addition, teachers need to be full participants in the decision-making process 
(Friend & Cook, 1990). Teachers also need support to plan and implement the interventions 
(Carta & Greenwood, 1997), as well as feedback and support to maintain intervention 
integrity (Noell, et al., 1997), and to monitor student progress (Shinn, 1995). Teachers need 
to be able to help decide what interventions are acceptable and will fit into their classrooms, 
as well as suggest the types of support they will need. Some teachers will need much more 
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training and feedback than others, and each teacher should be able to individualize this to 
some extent. 

All of these suggestions assume assistance team members have the necessary skills 
and time to provide these needed supports. Many schools currently lack one or both of these 
important components. The schools also need support to find additional resources, or to use 
their existing resources in more efficient ways. 

Another implication of this project is for training teachers and support personnel. 
Both groups of educators need to be fully informed about the importance of implementing 
and maintaining interventions with integrity throughout the entire intervention. General 
classroom teachers need more education to understand the role of progress monitoring in 
implementing interventions and how to use it effectively (Deno, 1997). Support personnel 
need skills in developing interventions that can be implemented with integrity by general 
classroom teachers, as well as how to provide support and performance feedback to teachers. 
Training for both groups needs to occur at the preservice and inservice levels. Even though 
teachers and support personnel have received some tr aining about these issues, they need 
more applied training (with performance feedback) to improve their skills. Preservice 
educators need assistance to be able to implement the needed skills from the very beginning 
of their career. 



Future Research 

The data presented here are one limited look at interventions for students in 
elementary school settings. However, these data serve as a be ginning to e xaminin g factors 
that influence intervention acceptability and integrity. The local observation study involved a 
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very limited number of teachers and students. Replication, with a larger sample of teachers, 
needs to be conducted to confirm the findings. 

Research studies examining intervention acceptability would also be improved if 
research could begin during initial planning stages so the acceptability of each proposed 
intervention could be evaluated. As much of the research about intervention acceptability has 
used analog studies, examining acceptability of proposed interventions for real student 
concerns would give more validity to the data. This would also allow for research 
examination of other participants’ involvement in the problem-solving process 
(administrators, assistance team members, parents, target students). These other participants 
could give their perspectives on the relative acceptability of proposed interventions, how to 
implement the steps discussed, and how to support the teachers during implementation and 
evaluation. 

Future studies could also examine the usefulness of written plans across more 
schools. Potential questions that could be addressed by research include: “Do teachers using 
written plans develop more effective interventions?”, “How do teachers develop effective 
plans?”, and “How specific do plans need to be in order to maximiz e effectiveness?” More 
work is also needed to understand how to incorporate effective progress monitoring into 
intervention planning and implementation. In addition, more research is needed about the 
role of progress monitoring in measuring intervention integrity, as well as other cost effective 
methods to measure integrity. 

These types of studies also need to be carried out across a wide variety of 
interventions and schools to determine if the findings are indicative of only one area, or are 
more reflective of the status of intervention implementation in general. Different states and 
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school districts may have policies and student populations that could radically change the 
type of interventions implemented, as well as results achieved. Analog studies were helpful 
to initiate research on this topic; now more studies conducted in actual school settings are 
needed. If teachers are expected to use effective intervention methods to work with students, 
then they need to have accurate information about the intervention planning and 
implementation process to do their job well. 
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APPENDIX A 

INITIAL INTERVIEW AND 
CLOSING INTERVIEW 
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Teacher Intervention Interview - Initial Interview 



Background 



1 . Current Position (Title) : 

2. Age: 

3. Gender: 

4. Number of years teaching (including the current year): 

5. Number of years in current position (including the current year): 

6. Total number of children in class: 

7. Number of children in class with written individualized intervention plans (not 

IEP’s): 

8. Number of children in class with IEP’s (individualized education plans in special 

education): 



General Questions 

1 . How do you deal with children having difficulty with learning and/or behavior in school? 



2. In what areas have you been the most successful helping students? 
Where have you had the most difficulty? 



3. What determines your decision to ask for additional help working with students? 



4. When you need extra assistance, how do you generally decide to seek outside assistance? 

What kind of assistance do you usually seek? 

From whom (principal, teachers, AEA, resources outside school, parents)? 

5. What can special education support personnel (AEA or district special education 
administration) do differently than they do now to help teachers? 
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6. When people talk about interventions for students, what comes to your mind? 



7. What changes, if any, would you like to see to help children with difficulties? 



8. What do you think is the best way to bring about change in how teachers work? 



9. What do you think is the best way to bring about change in schools? 



10. In your opinion, how helpful has inservice training (in the last three years) been in 
developing student interventions? 



1 2 


3 4 

1 1 


5 6 

1 1 


Not helpful 


Moderately helpful 


Very helpful 



"'"‘(List the most helpful topics). 



1 1 . In your opinion, how helpful have college or staff development classes been for 
developing student interventions? 

1 2 3 4 5 6 



Not helpful Moderately helpful Very helpful 

’•"•‘(List the most helpful topics —specify if the classes were staff development or 
college) 
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Teacher Intervention Interview - Initial Interview 



Intervention Specific 



1. Description of Concern 

A. What academic area is the concern? 

B. What is the goal of the intervention plan? 



1. Intervention Description 

A. What does it involve? What are the steps? 

B. Who is responsible for implementing the activities? 

C. What is the anticipated time line for the intervention to last? 




no 
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2. Intervention Planning 

A. How were you involved in the planning/decision making? 

B. Who else was involved? 

Were parents involved? 

C. Were you satisfied with your level of involvement/the involvement of others? 

D. How was the decision made to use this particular intervention? 

E. How much choice were you given in deciding on the specific intervention 
activities? 

Were you given the opportunity to accept/reject proposed plans? 

If yes, how? 

F. Would you have preferred a different intervention or strategy to implement? 

Why? 

In what ways would that intervention be different than the one you are using? 



3. 



Intervention Implementation 

A. What, if any material resources are needed to implement the intervention? What 
type? 

How do you get the resources, or the money for them? 

B. How much time is needed to implement this intervention? 

C. What type of skills are needed? 

D. How much effort is required by you to implement the intervention? By others? 




ill 
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4. Intervention Integrity 

Do you make sure the intervention is implemented as originally planned? 
How do you do this? 

Does anyone help with the effort? 



5. Intervention Monitoring 

A. How is student progress monitored? 

B. How often is the intervention reviewed for effectiveness? 

Does anyone help with the effort? 



6. Implementation Support 

A. Does anyone help you implement the intervention? 

B. Have any changes been made to your work load so you can implement the 
intervention? 

C. Does your administration support the intervention? 

If yes, how? 

If no, how do you know? 



7. Intervention Experience 

A. Have you had experience with problems like this before? 

If yes, how much experience? 

B. What has been most helpful to you with problems like this (the current and/or 

previous problems)? 

C. What has been the least helpful to you with problems like this (the current and/or 

previous problems)? 
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Teacher Intervention Interview (TII) — Closing Interview 



Background 



1 . Current Position (Title): 

2. Number of students in classroom/program: 



Intervention Specific 



1. Intervention Description 

A. What did the intervention involve? 

B. Who was responsible for implementing the activities? (List each person). 

C. When did the intervention begin, and how long did it last? 



2. Intervention Implementation 

A. Were any changes made to the intervention during implementation? 

If yes, what were they? 

Who decided the changes were needed? 

How involved were you in making this decision? 

B. What needed material resources different from those described in the initial 
interview? 

C. How much time was needed to implement the intervention? 

Was this different from those described in the initial interview? 

D. What type of skills were needed? 

Were these different from those described in the initial interview? 

E. How much effort was required by you to implement the intervention? By others? 

Were these different from those described in the initial interview? 



3. Intervention Integrity 

A. Did you implement the intervention as described in the written plan? 
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If yes, How did you make sure you implemented the intervention as written? 
If no, what changes did you make and why? 

B. Did anyone help with the effort? 



4. Intervention Monitoring 

A. How was the intervention monitored? 

B. How often did you review the progress monitoring data to evaluate student 

progress? 

C. Did you use progress monitoring data to change the intervention? 

Did anyone else review the progress monitoring data (instead of the you, or in 
addition to you)? 



5. Intervention Support 

A. Did anyone help you implement the intervention? 

B. Were changes made to your work load so you could implement the intervention? 



6. Intervention Outcome 

Was the intervention successful? Will it be continued? Will it be changed? 
What factors influenced the outcome? 



7. Study participation 

Do you think participating in the research study changed how you implemented the 
intervention? If so, how? 

Did completing the intervention checklists change how you implemented the 
intervention? 

Would you be willing to fill out the checklists even if you were not participating in a 
research study? 

Were the checklists helpful or a hindrance when working on the intervention? 
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APPENDIX B 



INITIAL INTERVENTION COMPARISON SCALE (IICS) AND 
CLOSING INTERVENTION COMPARISON SCALE (CICS) 




115 



103 



Initial Interview 



1 



Intervention Comparison Scale 



1 . How would you rate the severity of this child’s need? 

1 2 3 4 5 6 



Not at 




Moderately 


1 1 

Very 


all serious 




serious 


serious 


How suitable do you think this intervention is for this child’s need? 


1 

1 


2 

1 


3 4 

1 1 


5 6 

1 1 


Not at 




Moderately 


Very 


suitable 




suitable 


suitable 


Do you think this intervention targets the child’s most important need? 


1 

1 


2 

1 


3 4 

1 1 


5 6 

1 1 


Does not target 




Targets need 


Targets need 


need at all 




moderately well 


very well 


During intervention planning, was your input sought about possible intervention 


activities? 








1 

1 


2 

1 


3 4 

1 1 


5 6 

1 1 


No, 




Somewhat 


Yes, very 


not at all 






much so 


How would rate your involvement in the intervention planning process? 


1 

1 


2 

1 


3 4 

1 1 


5 6 

1 1 


No 




Moderate 


Very 


involvement 




involvement 


involved 


How would you rate your understanding of how to implement this intervention? 


1 

j 


2 

1 


3 4 

1 1 


5 6 

1 1 


Very 




Moderate 


Excellent 


poor 




understanding 





Initial Interview 



2 
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During intervention planning, were you given the opportunity to choose from 


proposed intervention plans? 




1 2 

L ' 1 


3 4 5 

1 1 1 


6 


Not at all 


Moderately so 


Very much so 


How much do you like the procedures used in this treatment? 


1 2 

j 1 


3 4 5 

1 I I 


6 

1 


Not at 


Moderately 


Very 


all 


well 


well 


How would your rate the extent to which your ideas were accepted and used during 


intervention planning? 






1 2 
1 1 


3 4 5 

1 1 1 


6 

1 


Not at all 


Moderately so 


Very much so 


How acceptable do you find this intervention for this child’s needs? 


1 2 
1 1 


3 4 5 

1 1 1 


6 

1 


Not at 


Moderately 


Very 


all acceptable 


acceptable 


acceptable 


How would you rate the amount of time this intervention will involve? 


1 2 
1 1 


3 4 5 

. 1 1 1 


6 

1 


None 


Moderate 


Excessive 




amount 


amount 


How would you rate the amount of materials and resources needed for the 


intervention? 






1 2 
1 1 


3 4 5 

1 1 1 


6 

| 


None 


Moderate 


Excessive 




amount 


amount 



Initial Interview 3 

13. To what extent are undesirable side effects likely to result from this intervention? 

1 2 3 4 5 6 
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None Moderate Excessive 

risk risk 

14. How effective do you think this treatment will be for this child’s need? 
1 2 3 4 5 6 



Not at Moderately Very 

all effective effective effective 

15. How much discomfort is the child likely to experience during the intervention? 
1 2 3 4 5 6 



None Moderate Excessive 

discomfort discomfort 



16. 



17. 



18. 



How likely is this intervention to make permanent improvements in the child? 



1 2 

1 1 


3 4 

1 1 


5 6 

1 1 


Not very 


Moderately 


Very 


likely 


likely 


likely 


How would you rate the practicality of this intervention given the amount of time 
required for planning the intervention? 


1 2 

j 1 


3 4 

1 1 


5 6 

1 1 


Not at all 


Moderately 


Very 


practical 


practical 


practical 


How would you rate the practicality of this intervention given the amount of time 
required for implementing the intervention? 


1 2 

J i 


3 4 

1 1 


5 6 

1 • 1 


Not at all 


Moderately 


Very 


practical 


practical 


practical 



Initial Interview 2 

19. How would you rate the practicality of this intervention given the amount of time 
required for record keeping? 

1 2 3 4 5 6 
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Not at all 
practical 



Moderately 

practical 



Very 

practical 



20 . 



How would you rate the amount of disruption this intervention will create for other 
students? 



1 2 

J L 

No 

disruption 



3 4 5 6 



Moderately Very 

disruptive disruptive 



2 1 . How would you rate the intervention in terms of the number of people needed to 

implement it? 

1 2 3 4 5 6 

J 1 1 I | l 

Excessive number of The number of people The number of people 

people are needed needed is acceptable needed is not a problem 
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Closing Interview 



1 



Intervention Comparison Scale 

1 . How would you rate the severity of this child’s need? 

1 2 3 4 5 6 



Not at Moderately Very 

all serious serious serious 

2. How suitable do you think this intervention was for this child’s need? 

1 2 3 4 5 6 



Not at Moderately Very 

suitable suitable suitable 



3. Do you think this intervention targeted the child’s most important need? 
1 2 3 4 5 6 



Not at Moderately Very 

all acceptable acceptable acceptable 

4. How would you rate your understanding of how to implement this intervention? 
.1 2 3 4 5 6 



Very Moderate Excellent 

poor understanding 

5. How much do you like the procedures used in this treatment? 

1 2 3 4 5 6 



Not at all Moderately well Very well 

6. How acceptable do you find this intervention for this child’s needs? 

1 2 3 4 5 6 



Not at Moderately Very 

all acceptable acceptable acceptable 



Closing Interview 



2 
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7. How would you rate the amount of time this intervention involved? 



1 2 
1 1 


3 4 

1 1 


5 6 

1 1 


None 


Moderate 


Excessive 




amount 


amount 


How would you rate the amount of materials and resources needed for the 


intervention? 






1 2 

1 1 


3 4 

1 1 


5 6 

1 1 


None 


Moderate amount 


Excessive amount 


To what extent did undesirable side effects result from this intervention? 


1 2 

1 1 


3 4 

1 1 


5 6 

1 1 


None 


Moderate risk 


Excessive risk 


How effective do you think this treatment was for this child’s need? 


1 2 
1 1 


3 4 

1 1 


5 6 

1 1 


Not at all effective 


Moderately effective 


Very effective 


How much discomfort did the child experience during the intervention? 


1 2 

J 1 


3 4 

i 1 


5 6 

_J l 



None Moderate discomfort Excessive discomfort 



1 2. How likely is this intervention in making permanent improvements in the child? 
1 2 3 4 5 6 



Not very likely Moderately likely Very likely 

13. How would you rate the practicality of this intervention given the amount of time 
required for planning the intervention? 

1 2 3 4 5 6 



Not at all practical Moderately practical Very practical 



Closing Interview 3 

14. How would you rate the practicality of this intervention given the amount of time 
required for implementing the intervention? 



0 
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1 2 3 4 5 6 



Not at all practical Moderately practical Very practical 

15. How would you rate the practicality of this intervention given the amount of time 
required for record keeping? 

1 2 3 4 5 6 



Not at all practical Moderately practical Very practical 

16. How would you rate the amount of disruption this intervention will create for other 
students? 

1 2 3 4 5 6 



No disruption Moderately disruptive Very disruptive 

17. How would you rate the intervention in terms of the number of people needed to 
implement it? 

1 2 3 4 5 6 

i 1 1 i i i 

Excessive number of The number of people The number of people 

people are needed needed is acceptable needed is not a problem 




122 



110 



APPENDIX C 

INTERVENTION OBSERVATION FORM (IOF) 
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Ill 

Classroom Observation 



| Behavior/Activity to be Observed * 1 

1 ! — — — — Z— . 






1. Date of observation: 2. Time of observation: 

3. Number of students in room: 4. Number of adults in room* 

5. Adult who conducts the intervention: 6. Length of intervention: 

Classroom Teacher (in minutes) 

Other Teacher (Specify: ) 

Regular Classroom Associate 
Special Associate 

Other (Specify: ) 

7. Intervention grouping: 

Whole Group (special assistance given during regular whole class instruction) 

Small Group (special assistance during regular small group instruction) 

Separate Small Group (group set up only for specialized assistance) 

Individual (adult works one-on-one with the r.hilH'i 
While the target student receives special assistance, other students are: 

Working in a Whole Group 
Working in Small Groups 
Working in Small Groups or Individually 
Working Independently 
, Not in Room 


1 feGJassrpb imEnyico amea^ 












1 . Classroom rules and procedures are clear. 


N 


S 


1 F 


UC 


2. There are interruptions in the flow of classroom activities. 


N 


S 


1 F 


UC 


3. Classroom is pleasant. 


N 


s 


F 


UC 


4. Classroom is productive. 


N 


s 


F 


UC 


5. Make transitions smoothly. 


N 


s 


1 F 


UC 


ea c n eriUehavao 












1. Teacher clearly explains what students are to do. 


N 


S 


F 


UC 


2. A variety of teaching methods and materials are used. 


N 


s 


F 


UC 


3. Teacher uses praise. 


N 


s 


F 


UC 


4. Teacher uses punishment. 


N 


s 


F 


r uc 


5. Teacher waits for students to respond. 


N 


s 


F 


UC 


6. Teacher provides instructional feedback about student responses. 


N 


s 


F 


uc 


7. Teacher monitors students. 


N 


s 


F 


uc 


mkargcvbt udehfiRartici 












1. Attends to teacher. _ J 


N 


s 


F 


uc 


2. Follows teacher directions. 


N 


s 


F 


uc 


3. Actively engaged in learning. 


N 


s 


F 


uc 


4. Answers questions when asked. 


N 


s 


F 


uc 


Key for Observation: N = Not occur S = Occurs Sometimes F = Occurs Frequently UC = Unc 


tear 
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APPENDIX D 

NATIONAL QUESTIONNAIRE 
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Teacher Questionnaire 
Background 

Please answer the following questions as completely as you can. When given a choice of responses, 
circle the number corresponding to your choice. 



1 . Current position (Title): 

1 = General education classroom teacher 

2 = Special education teacher 

3 = Special teacher (art, music, P.E., media, etc.) 

4 = Itinerant teacher 

5 = Other (specify: ) 

2. Your age: 

1 =20-29 

2 = 30-39 
3=40-49 

4 = 50-59 

5 = 60 + 

3. Gender: 

1 = Male 

2 = Female 

4. Grade(s) you currently teach: ; 

5. Number of years teaching (including the current year): 

6. Number of years teaching current grade (including the current year): 

7. Total number of children in your current class: 

8. Number of children with IEP’s in your class (individualized education plans in special education, 

even if special education services received elsewhere): 

9. Total number of students in building: 

10. Population size of community served by school district: 

1=0- 9,999 

2 = 10,000 - 49,999 

3 = 50,000 - 99,999 

4 = 100,000 or more 




Classroo 



1 * 6 . 



rventions 
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'assist students who find school 

difficult’ Interv entions refer to specific activities you, as the teacher, develop and use to help the I 
student with the particular difficulty he/she is having. ! 



1 1 . Please rate your experience in helping students having difficulties in the following areas. Use a 
scale of 1 to 3, where 1 means no experience and 3 means a great deal of experience. Use this 
same scale to rate your success in helping the students with the area of difficulty. 



Area of Intervention 


Experience 


Success 


None 


Some 


Great 

deal 


None 


Some 


Great 

deal 


Math 


1 


2 


3 


1 


2 


3 


Reading 


h 1 


2 


3 


1 


2 


3 


Written Language 


1 


2 


3 


1 


2 


3 


Other specific subject area. Specify: 


1 


2 


3 


1 


2 


3 


Study Skills 


1 


2 


3 


1 


2 


3 


Behavior Problems 


1 


2 


3 


1 


2 


3 


Other. Specify: 


1 


2 


3 


1 


2 


3 



12a. When working with students who have difficulty, how often do you obtain assistance from others 
to help you serve the students? 

1 = Never 

2 = Seldom 

3 = Sometimes 

4 = Often 

5 = Always 




12b. If you do obtain assistance, what type do you receive? On a scale of 0 to 3, please rate the 
helpfulness of each type of assistance you receive, where 0 means no assistance is received, 1 
means the assistance is not at all helpful, and 3 means it is very helpful. 



Type of Assistance 


Assistance 


None 


Not 

helpful 


Fairly 

helpful 


Very 

helpfUl 


Consultation with education specialists (e.g., special 
education teacher, school psychologist, etc.). 


0 


1 


2 


3 


Informal suggestions from other teachers. 


0 


1 


2 


3 


Extra personnel (e.g., more teacher associate time). 


0 


1 


2 


3 


Reduction in other duties. 


0 


1 


2 


3 


Materials (e.g., new/different books). 


0 


1 


2 


3 


Other. Specify: 


0 


1 


2 


3 


If you do obtain assistance, from whom do you receive i 


re assistance? Ratel 


he helpfulness 



of each person who provides assistance, where 0 means no assistance is provided by the 
person, 1 means the assistance is not helpful, and 3 means it is very helpful. 

I n 

4 ( 
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Assistance 


Assistance Provider 


None 


Not 

helpful 


Fairly 

helpfUl 


Very 

helpfixl 


Other General Education Teachers 


0 


1 


2 


3 


Principal 


0 


1 


2 


3 


Special Education Teacher 


0 


1 


2 


3 


Guidance Counselor 


0 


1 


2 


3 


School Psychologist 


0 


1 


2 


3 


School Social Worker 


0 


1 


2 


3 


Speech and Language Pathologist 


0 


1 


2 


3 


Other Special Education Support Personnel. Specify: 


0 


1 


2 


3 


Professionals outside the school. Specify: 


0 


1 


2 


3 


Other. Specify: 


0 


1 


2 


3 



13. Does your school expect teachers to work on individual interventions for students having difficulty? 

1 = Yes 

2 = No 

3 = Don’t Know 

14. Does your school have formal guidelines about what the interventions should include? 

1 = Yes What do these guidelines include? 

2 = No 

3 = Don’t Know 



Specific Interventions 



To answer the questions in this section, think of the student you have worked with most recently 
(in the past two years) who has had difficulty in your classroom. The student you select should 
have had learning and/pr behavioral needs that could not be met through the typical curriculum, 
and who is not receiving special education assistance. The questions ask about intervention(s) that 
you have used to assist the student in your classroom. 



15. In the past two years, have you worked on a intervention with a student having difficulty (who was 
not receiving special education services) in your classroom? 

1 = Yes Pick the most recent intervention and answer the following questions. 

2 = No — Please go to the comment section on the last page. 



16. 




What subject areas did the intervention address? (Circle all that apply). 

1 = Math 

2 = Reading 

3 = Written Language 

4 = Other specific subject area (list: 

5 = Study skills 



) 



best copy available 
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6 = Behavior problems 

7 = Other (specify: ) 

17. Did the intervention require that you change any of the following? (Circle all that apply). 

1 = Instructional materials 

2 = Instructional groupings 

3 = Instructional methods/approaches 

4 = Physical environment 

5 = Incentive (reinforcement) system 

6 = Other (specify: ) 

1 8. Was a specific plan (that included specified steps to follow when assisting the student) developed 
to meet the student’s needs? 

1 = Yes Was the plan documented in writing? 

1 = Yes 

2 =No 

2 = No What did you use to guide the intervention? 



19. How was the intervention developed? 

1=1 developed the plan alone 

2 = 1 developed the plan with the child’s parents 

3 = 1 developed the plan with informal consultation from another teacher/s 

4 = 1 developed the plan with formal consultation from another teacher/s 

5 = 1 developed the plan with special education support personnel (e.g., school psychologist, 

special education consultant, speech and language pathologist, etc.) 

6 = Other (specify: ) 

20. Who implemented the intervention? (Circle any that apply). 

1 = I did alone 

2 = Teacher associate/aide only 

3 = Other classroom teacher only 

4 = Special teacher only (specify: ) 

5 = Other special education support personnel only (e.g., school psychologist, special education 

consultant, speech and language pathologist, etc.) 

6 = Other (specify: ) 

2 1 . How many days did you work with the student on this intervention? 

1 = 1-5 days 

2 = 6-15 days 

3 = 16-30 days 

4 = 31-60 days 

5 = more than 60 days 



O 

ERLC 
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How many minutes per day did you work on this intervention (include time spent directly with the 
student, as well as planning time)? 

1 = 0-5 minutes per day 

2 = 6-15 minutes per day 

3 = 16-30 minutes per day 

4 = 31-60 minutes per day 

5 = more than 60 minutes per day 

23a. Was information gathered about student progress/change during the intervention? 

1 = Yes 

2 = No 

3 = Don’t Know 

23b. If Yes, who gathered the information? (Circle all that apply). 

1=1 did alone 

2 = Teacher associate/aide 

3 = Other classroom teacher 

4 = Special teacher (specify: ) 

5 = Other special education support personnel (e.g., school psychologist, special education 

consultant, speech and language pathologist, etc.) 

6 = Other (specify: ) 

23c. Did you make changes in the intervention as a result of student progress (because the 
student’s progress was better or worse than expected)? 

1 = Yes 2= No 

24a. Were efforts made to ensure the intervention was implemented as originally planned (intervention 
integrity)? 

1 = Yes 

2 = No 

3 = Don’t Know 

24b. If Yes, who ensured this? 

1=1 did alone 

2 = Teacher associate/aide 

3 = Other classroom teacher 

4 = Special teacher (specify: ) 

5 = Other special education support personnel (e.g., school psychologist, special education 

consultant, speech and language pathologist, etc.) 

6 = Other (specify: ’ ) 

24c. If you answered Yes to 24a, what did you do to ensure integrity of the intervention during 

implementation? 

1 = Someone observed me 

2 = Someone consulted with me 

3 = Used a checklist of the steps 

4 = Kept a calendar to indicate days the intervention was implemented 

5 = Kept progress monitoring data 

6 = No formal documentation 

7 = Other (specify: ) 
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Intervention Rating 

Please answer the following questions about the specific intervention procedure you selected in the 
previous section. For each item, circle the appropriate number. 



1. 


How acceptable did you find this 
intervention for this child’s needs? 


1 

1 


2 

_L_ 


3 

1 


4 

1 


5 

| 


6 

| 






Not at all 
acceptable 




Moderately 

acceptable 




Very 

acceptable 


2. 


How would you rate your 
involvement in the intervention 


1 

1 


2 

1 


3 

1 


4 

1 


5 

| 


6 

| 




planning process? 


No 

involvement 


Moderate 

involvement 




Very 

involved 


3. 


How would you rate your 
understanding of how to implement 


1 

1 


2 

1 


3 

I 


4 

I 


5 

| 


6 

| 




this intervention? 


Very 

poor 




Moderate 

understanding 




Excellent 


4. 


How would you rate the amount of 
materials and resources needed for the 


1 

1 


2 

1 


3 

1 


4 


5 

| 


6 

| 




intervention? 


None 




Moderate 

amount 




Excessive 

amount 


5. 


How would you rate the practicality 
of this intervention, given the amount 


1 

1 


2 

1 


3 

1 


4 

_J 


5 

| 


6 

| 




of time required for implementation? 


Not at all 
practical 




Moderately 

practical 




Very 

practical 


6. 


How would you rate the amount of 
disruption this intervention created 


1 

1 


2 

1 


3 

1 


4 

_J 


5 

| 


6 

| 




for other students? 


No 

disruption 




Moderately 

disruptive 




Very 

disruptive 


7. 


How acceptable was this intervention 
for your classroom/teaching style? 


1 

I 


2 

1 


3 

1 


4 

_J 


5 


6 

| 






Not at all 
suitable 




Moderately 

suitable 




Very 

suitable 


8. 


How effective do you think this 
intervention was for this child’s 


1 

1 


2 

1 


3 

1 


4 


5 

I 


6 

| 




needs? 


Not at all 
effective 




Moderately 

effective 




Very 

effective 


9. 


How would you rate your 


1 


2 


3 


4 


5 


6 



ERiC 
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10 . 



11 . 



12 . 



administration’s support of the 


1 


1 


1 1 


1 1 


intervention? 


Not at all 
supportive 




Moderately 

supportive 


Very 

supportive 


How would you rate the success of 
the intervention (i.e., meet the child’s 


1 

1 


2 

1 


3 4 

1 1 


5 6 

1 1 


needs so he/she could be successful in 
the classroom)? 


Not at all 
successful 




Moderately 

successful 


Very 

successful 


How much previous experience have 
you had with problems like this? 


1 


2 

1 


3 4 

1 1 


5 6 

1 1 


None 




Moderate 

amount 


A great deal 
of experience 


How likely are you to use this type of 
intervention again? 


1 

1 


2 

1 


3 4 


5 6 

1 1 


Not at all 
likely 




Moderately 

likely 


Very 

likely 



In closing, please provide any other comments about what you found to be helpful or not helpful 
when planning and implementing interventions for students. You may also comment on any i 
system-wide changes that address the above areas that you would like to see made. I 



Please us e the enclosed envelope to return the survey. Thank you for your assistance. 



120 



APPENDIX E 
FACTOR ANALYSIS 
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FACTOR ANALYSIS 

The 12 items from the Intervention Rating (IR) scale in the fourth section of the TIQ 
were submitted to a factor analysis to determine how well the items were measuring the 
concept of intervention acceptability. Cases were included in the analysis if the respondent 
answered at least 75% of the 9 of 12 scale items, resulting in a sample of 295 (only 
respondents who indicated they implemented an individual intervention in the last two years 
answered these items). Thirteen of these respondents did not answer all the IR questions, 
eight respondents answered 1 1 items, three answered 9, one answered 6 items, and one 
respondent did not answer any items. For respondents who answered nine or more items (11 
respondents), each individual’s average rating for the other scale items was substituted for 
the missing items. The two respondents who left out significant portions of the scale were not 
included in the factor analysis. 

As a first step in the factor analysis, the reliability of the scale was examined. Internal 
reliability was measured using Cronbach’s alpha (.83). However, examination of the item- 
total statistics indicated that one question (Question 4. “How would you rate the amount of 
materials and resources needed for the intervention?”) had a very low corrected item-total 
correlation (.12), and if the item were to be deleted the scale alpha would increase. 
Examination of the question showed the problem appeared to be in the response format. The 
response for this question did not have a linear increase from less to more positive as did the 
other questions (likert response format was l=None, 3 to 4=Moderate amount, and 
6=Excessive amount). An intervention that required a moderate amount of resources may not 
be viewed less positively than one that required no materials or resources to implement. As a 
result, this item was deleted from further analyses. Cronbach’s alpha for the new 1 1 item 
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scale was .85. Table 7 lists the correlations between the remaining 1 1 items in the 
Intervention Rating scale. See Appendix D (National Questionnaire) for a copy of the 
questionnaire to compare items. 

The next step in the factor analysis was to extract factors from the items. Several 
methods were explored for use, with the principal components analysis being selected as the 
best fit. Principal components analysis attempts to arrive at a small number of components 
that explains the largest amount of variance (Pedhazur & Schmelkin, 1991). Bartlett’s Test of 
Sphericity was conducted to determine if the correlations in the matrix were significantly 
different from zero (the matrix is not an identity matrix), with the result = 1465.81, p=<.001. 
This confirmed that the correlation matrix is not an identity matrix, and factor analysis could 
continue. 

The number of components to be included in the model was based on the criteria of 
eigenvalues greater than one. Both orthogonal and oblique rotation solutions were completed. 
The oblique method Direct Oblimin (Delta set at 0; SPSS, Inc., 1997) was completed first to 
determine if the factors were correlated with one another. This resulted in a three component 
model. An examination of the intercorrelations between the components showed that none 
reached .4, which was the level set as necessary to use the oblique solution (Pedhazur & 
Schmelkin, 1991). 

An orthogonal solution was then chosen as the final method for analysis. The 
Varimax orthogonal rotation method that simplifies interpretation of the factors was selected 
for this analysis (SPSS, Inc., 1997). This solution also resulted in three components that more 
clearly explained the components than the oblique solution (a criteria of a correlation of .4 
and above for inclusion on a component). E xaminat ion of the scree plot also indicates 



Table 7. Survey Intervention Rating scale item correlations. 
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existence of three factors (see Figure 4). The factor matrix can be found in Table 8. The 
components were Student Related Concerns, Practical Teacher Concerns, and Surrounding 
Factors. 



Student Related Concerns 

This component included six items and explained most of the variance of the model - 
44 %. Included were items directly related to the student targeted by the intervention (i.e., 
Acceptable for the child’s needs, Effective for this child’s needs, Success of the 
intervention), as well as several concerns that influence teacher availability to work with the 
student (i.e., Practicality given amount of time required for implementation, Fits 
classroom/teaching style, and Likely to use this intervention again). 

Practical Teacher Concerns 

This component included four items and explained 12% of the model variance. Items 
related to teacher concerns related to implementing an individualized intervention in the 
context of the other demands of teaching (Practicality given amount of time required for 
implementation. Amount of disruption for others, Fits classroom/teaching style, and 
Administration’s support) load on this component. Two of these items also loaded on the first 
component. 



Surrounding Factors 

This component included four items that accounted for 10% of model variance. The 
component label refers to items that surround implementation of the intervention 
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(Involvement in planning process, Understanding of how to implement the intervention. 
Previous experience with similar problems, and Likely to use this intervention again). One 
item also loaded on the first component. 

While three components were extracted from this analysis, the three are closely 
related and several items load on more than one factor, indicating that the scale might be 
considered to reflect an overall measure of teacher acceptability. As a result, the 1 1 items 
were summed for each person to create a total acceptability score used in further data 
analyses. 
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Table 8. Varimax factor matrix. 



Scale Item 


Factor 1 


Factor 2 


Factor 3 


1. How acceptable did you find this intervention 
for this child’s needs? 


.83 


.26 


.19 


2. How would you rate you involvement in the 
intervention planning process? 


.17 


.04 


.80 


3. How would you rate your understanding of 
how to implement this intervention? 


.10 


.37 


.77 


5 a . How would you rate the practicality of this 
intervention, given the amount of time 
required for implementation? 


.54 


.52 


.17 


6. How would you rate the amount of disruption 
this intervention created for other students? 


.12 


.79 


-.18 


7. How acceptable was this intervention for your 
classroom/teaching style? 


.43 


.68 


.25 


8. How effective do you think this intervention 
was for this child’s needs? 


.89 


.19 


.17 


9. How would you rate your administration’s 
support of the intervention? 


.13 


.60 


.16 


10. How would you rate the success of the 
intervention (i.e., meet the child’s needs so 
he/she could be successful in the classroom)? 


.90 


.11 


.14 


1 1. How much previous experience have you had 
with problems like this? 


.14 


-.09 


.56 


12. How likely are you to use this type of 
intervention again? 

a a . • i t * 71 ; : 


.55 


.33 


.52 



a Item 4. was not included in the analysis. 
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